MFP language Help: Game programming

MFP language Help: Image and Sound Processing in Game Development

In the previous section animation principle has been introduced. However, without optimization, repainting every object on display window in each animation step is very expensive and makes a game sluggish and irresponsive. Provided that accessing RAM is much faster than reading/writing physical screen (i.e. video memory), a better solution would be drawing all objects in an image, and then pasting the image to display window. In this way, image processing replaces video memory reading/writing so that animation is much more swift and responsive.

Moreover, sound effects are always required when playing a game. This section will also address sound processing.

1. Create, Load, Clone and Save Image

The following functions create, load, clone and save image, return image size and validate image handle respectively.

Function Name	Function Info
create_image	::mfp::multimedia::image_lib::create_image(2) : create_image(w, h) returns a new and blank wrapped JAVA image object with width = w and height = h.
load_image	::mfp::multimedia::image_lib::load_image(1) : load_image(image_path) returns a wrapped JAVA image object. image_path is a string based path pointing to an image file.
load_image_from_zip	::mfp::multimedia::image_lib::load_image_from_zip(3) : load_image_from_zip(zip_file_name, zip_entry_path, zip_file_type) returns a wrapped JAVA image object loaded from a zipped file. Its first parameter is the path of the zipped file. Its second parameter is the zip entry path of the image. Its last parameter is zip file type. 0 means it is a normal zip file and 1 means it is an Android asset zip file (for MFP app).
clone_image	::mfp::multimedia::image_lib::clone_image(7) : clone_image(image_src, src_left, src_top, src_right, src_bottom, dest_width, dest_height) returns a new wrapped JAVA image object with width = dest_width and height = dest_height. The returned image is a (zoomed) copy of image_src's selected area. The selected area's left, top, right and bottom are src_left, src_top, src_right and src_bottom respectively. Note that src_left, src_top, src_right and src_bottom are optional. By default, they equal 0, 0, width of image_src and height of image_src respectively. dest_width and dest_height are also optional. By default, dest_width equals src_right - src_left and dest_height equals src_bottom - src_top. An example of this function is clone_image(img_src, 0, 0, 100, 200, 50, 300) .
save_image	::mfp::multimedia::image_lib::save_image(3) : save_image(image, file_format, path) saves a wrapped JAVA image object to an image file. The first parameter is the wrapped JAVA image object. The second parameter is the format of the file. It is a string and currently only "png", "jpg" and "bmp" are supported. The third parameter is the path of the file. This function returns true if the file is saved successfully, and returns false if any failure. For example, save_image(img, "png", "C:\\Temp\\1.png") .
get_image_size	::mfp::multimedia::image_lib::get_image_size(1) : get_image_size(image_handle) returns size (i.e. [width, height]) of a wrapped JAVA image object (i.e. image handle).
is_valid_image_handle	::mfp::multimedia::image_lib::is_valid_image_handle(1) : is_valid_image_handle(image_handle) returns true or false, telling developer if a image handle (i.e. a wrapped JAVA image object) is still valid or has been closed.

2. Modify Image

Before pasting an image onto display window, developer generally needs to modify it, i.e. painting on top of the image. In MFP, APIs to paint on image are either the same as or very similar to APIs for display window. Developer calls function open_image_display to create an “image display”. This display’s original size is the same as the image and its size can also be adjusted by set_display_size function. The image itself is the background image of the “image display”. Exactly same as display window, developer is able call functions like draw_rect and draw_image to paint on the “image display”, and call functions update_display and drop_old_painting_requests to control painting time and manage painting events respectively. After drawing finishes, function get_display_snapshot has to be called to obtain the snapshot of the “image display”, i.e. the image after modification. This snapshot can be used by function draw_image to be “pasted” onto a display window. When the image’s updated snapshot is no longer used, developer can call shutdown_display to close the “image display”.

Developer may keep in mind that, first of all, painting on an “image display” will not change original image. In other words, after updating an“image display” and then closing it, its original image is still unchanged. Second, similar to physical display window, if an “image display”’s painting event scheduler includes too many painting events, updating it will be very slow. A solution is to call function set_display_snapshot_as_bgrnd. This function uses snapshot of a display, whether a display window or an “image display”, as the display’s background image. Therefore, all the painting events to draw the new background image can be removed after the new background is created.

The details of the above functions are listed as below:

Function Name	Function Info
open_image_display	::mfp::multimedia::image_lib::open_image_display(1) : open_image_display(image_path_or_handle) creates an image display for developer to paint. image_path_or_handle is either a string based path pointing to an image file, or null, or a memory handle of a JAVA image object returned by load_image, load_image_from_zip, create_image or clone_image functions.
get_display_snapshot	::mfp::graph_lib::display::get_display_snapshot(4) : get_display_snapshot(display, update_screen_or_not, width_ratio, height_ratio) returns a display (whether screen display or image display)'s snapshot. Its second parameter, update_screen_or_not, telling MFP whether or not the display should be refreshed so that latest image can be captured. The third and fourth parameters are optional. They tell MFP how to zoom the snapshot. By default, both of them are 1. For example, get_display_snapshot(d, true, 0.5, 3) refreshes display d (i.e. all the painting event callbacks take effect) and then takes snapshot of the display and returns a zoomed image. The width of returned image is 0.5 * original snapshot width and the height of returned image is 3 * original snapshot height.
set_display_snapshot_as_bgrnd	::mfp::graph_lib::display::set_display_snapshot_as_bgrnd(3) : set_display_snapshot_as_bgrnd(display, update_screen_or_not, clear_callbacks_or_not) sets a display (whether screen display or image display)'s snapshot as background image. Its second parameter, update_screen_or_not, telling MFP whether or not the display should be refreshed so that latest image can be captured. The third parameter, clear_callbacks_or_not, tells MFP whether or not the painting event callbacks should be cleared. For example, set_display_snapshot_as_bgrnd(d, true, true) refreshes display d (i.e. all the painting event callbacks take effect) and then drops all the painting event callbacks, and then takes snapshot of the display and sets the snapshot to be display's background image.

The following code is copied from the Hungry Snake game. It uses “image display” to draw static objects. And then the snapshot of the “image display” is used as display window’s background image. This approach avoids overhead paintings in each animation step.

// open an empty image display

variable boardImageDisplay = open_image_display(null)

// adjust it's size to game display window's size times scaling ratio

set_display_size(boardImageDisplay, windowWidth * scalingRatio, windowHeight * scalingRatio)

// calculate text origin of level information

variable textOrigin = [10, 10]

variable levelFontSize = LEVELFONTSIZE()

if xMargin > yMargin

// text is in the center of left edge rectangle

textOrigin = calculate_text_origin(DISPLAYSURF, "level " + level, [0, 0], xMargin, windowHeight, 0, 0, levelFontSize)

else

// text is in the center of top edge rectangle

textOrigin = calculate_text_origin(DISPLAYSURF, "level " + level, [0, 0], windowWidth, yMargin, 0, 0, levelFontSize)

endif

// draw level information text. Note that the text is scaled down to fit the image.

draw_text("static element", boardImageDisplay, "level " + level, textOrigin * scalingRatio, scoreColor, levelFontSize * scalingRatio)

// draw the border of snake's moving space. Note that the rectangle is scaled down to fit the image.

draw_rect("static element", boardImageDisplay, [xMargin, yMargin] * scalingRatio, gridWidthDim * scaledCellSize, gridHeightDim * scaledCellSize, boarderColor, 1)

// draw the wall. Note that the wall is scaled down to fit the image.

drawPoints("static element", boardImageDisplay, wallPlace, wallColor, cellSize, xMargin, yMargin, scalingRatio)

if(shouldDrawButtons)

// we draw text only for each button because button text is static while button border is not.

drawButtonText(boardImageDisplay, upBtnLT, btnW, btnH, "Up", false, scalingRatio)

drawButtonText(boardImageDisplay, downBtnLT, btnW, btnH, "Down", false, scalingRatio)

drawButtonText(boardImageDisplay, leftBtnLT, btnW, btnH, "Left", false, scalingRatio)

drawButtonText(boardImageDisplay, rightBtnLT, btnW, btnH, "Right", false, scalingRatio)

endif

// get snapshot of the image display, note that we update the image display before taking snapshot

variable boardImage = get_display_snapshot(boardImageDisplay, true)

// shutdown image display

shutdown_display(boardImageDisplay)

// set the snapshot of the image display to be game's display window's background image.

// note that the mode is stretching the background image to fit the whole game's display window

// as the snapshot image is smaller than the game's display window.

// 游戏真实显示窗口大小一致。

set_display_bgrnd_image(DISPLAYSURF, boardImage, 1)

3. Draw Images

Function draw_image add an image-painting event into painting event scheduler. This function can be called in two different ways. The first one is draw_image(owner_info, display, image_or_path, left, top, width_ratio, height_ratio, painting_extra_info). The second one is draw_image(owner_info, display, image_or_path, srcx1, srcy1, srcx2, srcy2, destx1, desty1, destx2, desty2). In both of the calling approaches, the first parameter is owner_info. Owner_info tells painting event scheduler who owns this painting event. Owner_info can be a string, which means the name of the owner. It can also be an integer, meaning the id of the owner. It can even be NULL, which means system owns the painting event. Owner_info may also be a two element array. The first element of the array is a string (i.e. owner’s name) or an integer (i.e. owner’s id) or NULL (i.e. system). The second element of the array is a floating value working like a time stamp. However, it is not a real time stamp. It can be any value. It will be used when the scheduler starts to clear painting events. The second parameter of the above calling approaches is display window’s handle. The third parameter of the above calling approaches is the handle of a image or a string based image file address. The last parameter, painting_extra_info, tells scheduler what is the porterduff mode to draw the target image. This parameter is optional. The mechanism of porterduff mode is very complicated. So default value (i.e. ignoring this parameter) is strongly recommended. For further details, developer may refer to related JAVA documents.

In the first calling approach, the fourth to the seventh parameters are, respectively, left of the painting destination, top of the painting destination, .scaling ratio along the x-axis (optional, by default it is one), scaling ratio along the y-axis (optional, by default it is one). In the second calling approach, the fourth to the eleventh parameters are, respectively, left of the painting source, top of the painting source, right of the painting source, bottom of the painting source, left of the painting destination, top of the painting destination, right of the painting destination and bottom of the painting destination. Examples of draw_image function are draw_image("image", display, get_upper_level_path(get_src_file_path()) + "gem4.png", 48, 157), draw_image("image", display, gem3Img, 148, 257, 3, 0.5) and draw_image("imagesrc", display, gem3Img, 0, 0, 32, 32, 210, 540, 300, 580, a_painting_extra_info).

4. Play Sound

MFP has the following sound processing functions:

Function Name	Function Info
play_sound	::mfp::multimedia::audio_lib::play_sound(4) : play_sound(source_path, repeat_or_not, volume, create_new_or_not) plays a sound file (can be wave, midi or mp3). This function returns a sound handle which is a JAVA (or Android) media player wrapper. Because the media player resource is scarce, this function will try to reuse previously created sound handle. It has four parameters. The first parameter is the path of the sound file. The second parameter is a boolean telling MFP whether the sound should be repeated to play or not. This parameter is optional and its default value is false. The third one, which is a double from 0 to 1, is the volume of the sound. This parameter is optional and its default value is 1. The fourth one is a boolean flag telling MFP to create a new sound handle compulsorily or not. If it is true, play_sound always creates a new sound handle. This function is optional and its default value is false.
play_sound_from_zip	::mfp::multimedia::audio_lib::play_sound_from_zip(6) : play_sound_from_zip(source_zip_file_path, zip_entry_path, zip_file_type, repeat_or_not, volume, create_new_or_not) plays a sound file (can be wave, midi or mp3) extracted from a zip file. This function returns a sound handle which is a JAVA (or Android) media player wrapper. Because the media player resource is scarce, this function will try to reuse previously created sound handle. It has six parameters. The first parameter is the path of the zip file. The second parameter is zip entry path of the zipped sound file. The third parameter is zip file type, 0 for normal zip file and 1 for MFP app's Android asset zip file. The fourth parameter is a boolean telling MFP whether the sound should be repeated to play or not. This parameter is optional and its default value is false. The fifth one, which is a double from 0 to 1, is the volume of the sound. This parameter is optional and its default value is 1. The sixth one is a boolean flag telling MFP to create a new sound handle compulsorily or not. If it is true, play_sound always creates a new sound handle. This function is optional and its default value is false.
start_sound	::mfp::multimedia::audio_lib::start_sound(1) : start_sound(sound_handle) plays a sound referred by sound handle sound_handle. If the sound has been started, this function will do nothing.
stop_all_sounds	::mfp::multimedia::audio_lib::stop_all_sounds(0) : stop_all_sounds() stops all playing sounds.
stop_sound	::mfp::multimedia::audio_lib::stop_sound(1) : stop_sound(sound_handle) stops the playing sound referred by sound_handle. If the sound is not playing, it does nothing.
get_sound_path	::mfp::multimedia::audio_lib::get_sound_path(1) : get_sound_path(sound_handle) returns sound file path of the sound handle.
get_sound_reference_path	::mfp::multimedia::audio_lib::get_sound_reference_path(1) : get_sound_reference_path(sound_handle) returns sound reference file path of the sound handle. If the sound is not extracted from a zip file, sound reference file is the same as sound file. If the sound is extracted from a zip file, sound reference file path is the zip file path followed by the sound's zip entry path, like "/folder1/folder2/snd.zip/zipped_folder/snd.wav", where "/folder1/folder2/snd.zip" is path of the zip file and "zipped_folder/snd.wav" is zip entry path.
get_sound_repeat	::mfp::multimedia::audio_lib::get_sound_repeat(1) : get_sound_repeat(sound_handle) returns a boolean indicating the sound is repeatedly played or not.
get_sound_source_type	::mfp::multimedia::audio_lib::get_sound_source_type(1) : get_sound_source_type(sound_handle) returns an integer which is sound reference source file type. 0 means normal source file, 1 means zipped source file and 2 means zipped source file in Android asset (for MFP app).
get_sound_volume	::mfp::multimedia::audio_lib::get_sound_volume(1) : get_sound_volume(sound_handle) returns volume of the sound referred by sound_handle. Volume is a double value ranging from 0 to 1.
set_sound_repeat	::mfp::multimedia::audio_lib::set_sound_repeat(2) : set_sound_repeat(sound_handle, repeat_or_not) set a sound handle to play repeatedly or not.
set_sound_volume	::mfp::multimedia::audio_lib::set_sound_volume(2) : set_sound_volume(sound_handle, volume) set volume (from 0 to 1) a sound handle.

Because of limited resource, MFP tries to reuse existing media players. MFP compares reference path of a to-be-played sound file with reference path of each existing media player. If there is a match, MFP reuses the existing media player. Otherwise, MFP creates a new media player.

In most cases, reference path of a sound file is the same as its actual path. However, if a sound file is read from a zip file (including a zip file in MFP apps’ asset), this sound file will be copied to a temporary place in SD card or hard drive. In this case, the temporary place is the (actual) path of the sound file, the reference path is the zip file path followed by zip entry path of the sound file. For example, in reference path "/folder1/folder2/snd.zip/zipped_folder/snd.wav", zip file path is "/folder1/folder2/snd.zip", zip entry path is "zipped_folder/snd.wav".