Tuesday, 26 June 2012

A real saviour

one thing that is rather common among all adventure games, is not only the ability, but also (without naming names) the NEED to save early and save often, certain games could even make good use of a version control system for that particular thing...

Oh well, puns and jokes aside, last week I added savegame-support in the WME-port, initially this worked fine for everything BUT J.U.L.I.A. and after looking for the problem for a good day, I noticed that the savegame-code for the theora-wrapper had a special case for savegame-versions below a certain threshold, thus reading fewer bytes than was written. (And well, I had reset the savegame-version system to 1.0.0 at the time, since I wasn't aiming for savegame-compatibility anyhow). Fixing that allowed for working savegames in all the games I have tested so far (which would be Dirty Split and J.U.L.I.A.).

I also made some headway with the renaming of functions the past few days, now a good chunk of the functions follow the "funcName()"-convention, instead of the "FuncName()"-convention. A few still remain, and the arguments and variables still stay mostly unchanged.

I also finally got around to compiling the port on a PPC Mac, only to realize how much worked there: the mouse cursor. Yup, some Big-Endian-love is needed to get this train rolling on such CPUs. The main problem at the moment lies in how the scripts are read: *(uint32*)char_buffer... I'm thinking I'll replace most of that code with MemoryStream-code, to make the endian-handling a tad easier (and possibly avoid having to read the entire file in one big chunk at game-start.

The persistence system used in WME originally made a note of the saveable classes by having a static instance of CSysClass per class, this was a bit too global for my tastes, and I have changed that to a function that manually registers all the classes, this way it can be cleaned up properly on exit.

Along with these changes, I also added in load-from-launcher support, and RTL-support, right now the thumbnails are a bit off for load-from-launcher, but the same problem exists for ingame thumbnails too, I guess I'm using a bit too much of the render-surface for the thumbnail, either that, or the downscaling works a bit quirkily.

I'm hoping to have atleast the scripts rolling on PPC this week.

Sorry for the lack of exciting pictures this time, but atleast this means both your game progress and bandwidth can be saved now. 



Thursday, 14 June 2012

10 days later

I left the last blogpost with an image of a video from J.U.L.I.A. being played inside ScummVM, but that image still didn't have the clearing of the screen included, so to start of this post, here is how it currently looks:
The J.U.L.I.A.-demo intro

That improvement did however exist at the time I posted that last blogpost, so let's get on with what has improved since then:

TTF-Rendering looks better

Last time I posted a screenshot of the TTF-rendering that looked like this:
How fonts looked last week
Notice that the text is impossible to read, as well as being quite small. There was also an issue with the size of each line making the "descending" parts of characters cut off on the lowest lines. Thanks to a bit of help from LordHoto, I managed to find out that I had been stupid, and used the wrong variables for creating the drawing-surface for the fonts, I had used the font height that the scripts ask for, while in reality I should have used the font height that the TTF-loader in ScummVM returns (which is the height from the lowest point to the highest point, instead of the height from the baseline to the highest point).

While looking through the code to find out why this happened, I took this screenshot, which shows red lines at the points I was asking the TTF-font to render each line:

Why using the wrong height looks wrong.
The fonts were also drawn about too small, as WME used a dpi that was 4/3 of what ScummVM uses in it's TTF-code (96 vs 72), simply multiplying the font-size specified by the scripts made the font a lot better, although, there were still some issues relating to the "layering" that WME does to get the shadowed-fonts it has. As it turns out, this was done by drawing the same line multiple times with different colour-values, at differing offsets. The current draw-code did something rather silly when it got offsets (well, it simply changed the left and top parts of the rect that specifies the draw-area, without making sure the width stayed the same, making it an odd skew-and-scale-operation). Fixing that as well provided the following result:
Font-drawing after fixing offsets and sizing.
Making sure we even bother to use the text alignment the scripts ask for, as well as mostly the right colours, provides quite an improvement for Dirty Split too, compare last posts font-rendering:
Last week's rendering
To this weeks version of the exact same scene:
This weeks rendering
It's not perfect, as the text should be a bit lighter, but it's getting there atleast.

String-concatenation works again:

... and I didn't even know I broke it, well, turns out I must have replaced some variable-names in the II_ADD-script-function, and forgot to make sure everything was changed correctly, which made concatenation of strings always return NULL instead of what it was supposed to return, thus making things like SetImage completely broken, putting that right again, provided one of the biggest leaps of functionality I've seen in a while:
What difference one single variable-name can have.
 Notice that the images on the buttons are in place now in the above picture.

Entity-videos now work too

I forgot that I had to add a few bits and pieces to AdEntity too, to get the videos working for non-fullscreen scenes like this one:
Waking up all alone in the middle of nowhere, literally.
Those little videos also use alpha-masks to allow for non-rectangular videos, in this case, the face in the upper right corner has such a mask to hide the parts that would otherwise fit in the rectangular box it's being played in:
Alpha-masked video
Note though that the current solution for applying alpha-masks is both ugly, and non-endian safe, and was mostly put in to make sure things looked right, I will be cleaning this up in the coming weeks.

As a final comparison: Look at this picture from last week:
Last week's "Enter manual override"
Now compare that to:
This week's version of the same scene
In the upper picture you can see the rectangular box that the videos are being played in, that is hidden by the alpha masking now. Also note the difference in the menu-buttons, and the alignment and presentation of the text. You'd almost think that the demo was completable now...

And it is:

After fixing all of the above, and getting rid of a few hundred megabytes of memory leaks, the game still hung when going down to the planet, so I started looking, and it turns out that the string-splitter function had a bug that caused it to never move on from the first token, but simply continued to push that token to it's result-list, thus eating up a few gigabytes in a matter of seconds. Replacing the STL-string-like solution with ScummVM's Common::StringTokenizer, fixed the issue, and allowed me to complete the planet-side mission, thus letting me complete the last two puzzles.
The last puzzle in the J.U.L.I.A.-demo running in ScummVM

 In the tradition of showing something that DOESN'T work as a closing-point, here's a picture of how J.U.L.I.A. looks when closing the game during the tutorial...


Monday, 4 June 2012

A show and tell in 10 steps

If the last post was worth a few thousand words, on account of it's images, then this one might be worth quite a lot more than that, as I've had quite some progress in the visual parts of the Wintermute-engine, without further ado, ten improvements from the last post:

1: Alpha-blending now works properly

As you might remember from my last post, alpha blending was sort of non-existent in the code at that point. I did do specific skipping of completely transparent pixels, but that didn't prove to pretty:
Previous version, without alpha-blending for anything but full transparency
_sev commented on my last blogpost:
"I recommend to take a look at the code of RenderedImage::blit() in engines/sword25/gfx/image/renderedimage.cpp" 
Which I did, and although I haven't completed the refactoring into common-code that he asked for yet, I did add the code to my own branch, which gives the following result:
New version, with alpha transparency
 Notice that the trees look a lot better, along with the bars on the wall. The hair is also less jaggy, although that is not that visible in this small screenshot. A little downside about the current solution is that it is quite CPU-intensive. But that's a problem for another day, for now it atleast works properly.

2: Bitmap-fonts are drawn correctly now

Previously, the bitmap-fonts were drawn with some rather weird offsets, making the characters have odd spacing. The transparency in those images were also quite off:
Previous version, with broken bitmap-fonts
Now, these bitmap-fonts are BMPs, which have rather quirky alpha-channels, the original WMELite-code had this comment hanging around:
// convert 32-bit BMPs to 24-bit or they appear totally transparent (does any app actually write alpha in BMP properly?)
Which explained that part of the problem, I clearly had to replace the existing alpha-channel with the colour key specified by the game.

As for the weird spacing, the engine uses BSurface::IsTransparentAtLite to detect the width of the characters. This is also the reason why the engine explicitly requires the characters to be left-aligned.
Now IsTransparentAtLite was one of the functions that were stubbed when I removed the SDL2-requirement, so adding in new code to check for transparency solved that problem:

New version, with working bitmap-fonts.

3: Sprite scaling works

If you look at that last picture, do you notice anything else that's different from the previous ones? Yeah, Baxter is a bit smaller, and for a good reason, the code I borrowed from the Sword 2.5-engine included scaling before drawing, which solves quite a few issues. Previously if you tried to walk up to the door, Baxter would walk in-place, and not seem to go anywhere, simply because he couldn't "walk into the screen" in any visible fashion (he would have had to get smaller to show it). But now, he's scaled accordingly. That screenshot is actually quite close to this screenshot from the original WME Lite-engine, that you might remember from my last post:
The original WME Lite-engine

Yes, there are a few minor differencies, for one, there's some definite breakage in the gradient that makes up the road he's standing on, but the important bit here is, that this is the very first scene in the game, without any movement done, if you compare this to the first two images in this post, notice how he was way to big to begin with? Well, that's solved now.

4: Sprite mirroring works

One of the features that were missing from WME Lite, when compared to the full Wintermute-engine, was sprite mirroring/rotation, since the Sword 2.5 code I borrowed also contained support for vertical/horizontal flips, it was rather straight-forward to add that little feature back in. Now for testing this, I had to find a game that used it, which in this case was the game Rosemary. Walking left in this game was done by simply mirroring the walk-right sprites, which made it look rather odd without sprite mirroring enabled:
Walking left in Rosemary, without sprite-mirroring
 Now, simply hooking the right data back up to the blit-code from Sword 2.5, made this look way better:
Walking left in Rosemary, WITH sprite-mirroring

5: Fixed some bugs that crept in while refactoring code

Yes, bad things might happen when handling large code-bases, back before the last blog post, I changed a lot of char* into const char* to silence some warnings, and ease the hunting of a few segfaults, this had the interesting side effect of breaking a few virtual-functions, making inheriting classes no longer override the proper functions (C++ can be tricky there, as const char* and char* are quite different types that can be cast only one way). This led to scripts like playRandomMouseOver in Rosemary not running, as the super-class version got run instead of the correct one (the reasoning behind the bug was simple; the function-name was stored in a char* and the super class declared a function that accepted char*  as parameter, while the subclasses accepted const char*, thus the wrong one was chosen).

I also had a few issues with another silly bug: After adding in handling of key-presses, every key seemed to trigger the exit-game dialogue. After hunting that one for quite some time, I realized that I'd made a silly mistake in the event-handling in PlatformSDL.cpp,  right after the case for case Common::EVENT_KEYDOWN: was case Common::EVENT_QUIT: and... there was no break; between them...

6: Fixed speaking to characters (and in doing so, most of the sound)

Previously, if you tried to talk to any character in Dirty Split, the conversation would be cut short, this was because the engine needs to know the length of the text, to know how long it should wait before issuing new lines of text, and unlocking the objects that are speaking. When I did the sound-system before the last blog-post, I only added in the necessary bits for getting something to play, but I didn't bother doing all the various other functions. Specifically GetLength() proved to be quite important for this little tidbit, as it was previously just stubbed to 0. Explaining the rather short conversations.

What remains to be done in the sound-system, is (among other smaller things) supporting loops and seeking in the audio.

7: Added detection for a few more games

I changed the detection-scheme to use the Advanced Detector so that I could easily flag the various games for how compatible they were, as well as detect them based on more detail than the original detection solution. Now, adding every single game to this list will be a never-ending job, as new games are made all the time, but at least the known good and known bad games should be added over time. Particularly the known bad ones should be important to add here (such as all the 3D-games), so that the users will know why their particular game won't work. Another thing planned for this solution, is to have a simple fallback that registers unknown games by simply finding a data.dcp file that doesn't match any known ones. This should allow playing any new or in-development titles without having to first put the hashes into ScummVM.

The currently added games are:
I'll happily add more, if anyone has any good suggestions, but these at least let me test a few of the functions in the engines rather thorougly.

8: TrueType-font support

I have started working on allowing the games to load the TTF-fonts they ask for, which is rather necessary to for instance get subtitles in the conversations in Dirty Split. WME does support TTF-fonts, and defaults to using Arial, this does pose a problem for a ScummVM-port though, as while Arial can be loaded easily on Mac OS X or Windows, from the system-font-directories, various other platforms either lack Arial, or even a system font-concept at all. I'm looking at finding a way to use GNU FreeFont as a fallback alternative for these platforms.

There is also the ever-existing question of what do to if FreeType2 isn't available when building, I guess there will have to be some sort of fallback to one of the GUI-fonts in ScummVM, as a fallback in that case.

Anyhow, TTF-support is still a bit lacking in the WME-port at the moment, as I haven't entirely worked out the details for getting the proper sizes for the fonts, but that's work that will be solved in the coming weeks.

For the curious, this is how TTF-font-rendering looks in WME Lite:
TTF-font in WME Lite
And this is how it currently looks in ScummVM:
TTF-font in ScummVM

For comparison's sake, both of these use Arial at the moment, but long term, if users want to have Arial or something similar used with their games, they'll have to put a copy of the font file in their game folder.

One of the things that are handled fine at this point though, is text-wrapping:
Text-wrapping in J.U.L.I.A.
Yes, it's ugly at the moment, and the picture above even uses a few cheats to get the lower parts of the characters to get drawn (think the "cellar-part" of a g for instance), but it is what it is, a work in progress. 

9: Screen refreshing now works

The original engine cleared the screen to black before every frame, that was another part of the SDL-code that I removed while getting it compiling in ScummVM. At certain points this would mean that parts of the screen that were supposed to be black, just kept what was there in the last frame that updated it. Which isn't really pretty:
J.U.L.I.A.-demo without clearing
Especially the particle engine in the above shot gives use problems. Anyway, making the entire screen draw a color between frames made things quite a lot prettier:

J.U.L.I.A.-demo with clearing
The downside of this though, is that the blitting functions used here aren't exactly free CPU-wise, so having to redraw everything every frame doesn't exactly reduce the CPU-usage. But atleast J.U.L.I.A. looks right now.

10: Theora-video-support

Now here comes the reason why I was talking about images in the opening of this post, because, if one image is worth a thousand words, then full motion video must be worth rather vastly larger sums than that again.

I mentioned J.U.L.I.A. in #9, and that's a game that really needs video support to shine. I got bored of hunting a bug at some point during the past few days, and decided to do something else for a while, which meant adding back Theora-video-support to the engine.

This was yet another field where the Sword 2.5-engine came to the rescue, as it already had a ScummVM-friendly implementation of libtheora, ready for use as a VideoDecoder, which meant that all I had to do was download the original WME sources (after contacting Mnemonic to make sure I could use them under GPLv2, instead of GPLv3), and dig out the removed pieces of code from there. In this case, that meant the code for functions like PlayTheora in AdEntity/AdGame, as well as the Vid-files from Base/ to see what the interface had been originally.

Connecting up the Sword 2.5 wrapper for libtheora to the WME-theora handler wasn't much work, and soon I had video's playing in all my games. A current limitation though, is that seeking is not yet implemented for Theora.

I'll close of this rather lengthy blogpost, with an image of a video playing in J.U.L.I.A.:
The intro video from the J.U.L.I.A.-demo
(note that this image was taken before fixing the screen-clearing issue)

Tuesday, 22 May 2012

A picture says more than a thousand words


...which should put this blog entry far beyond 10 000 words. There has been a bit of progress with the Wintermute-engine, preliminary support for graphics, sound and I/O is in place, but it is far from perfect, or finished in any way, shape or form.

Right now, the sound system only allows playback of OGG-files, and it only does Play() for them, which was a rather simply task to add in, simply to verify that the sound system still works after having been stripped down from WinterMute Lite, either way no further work is planned on that part for atleast a few weeks.
I also added in some quick hacks to make the engine draw inside ScummVM, right now this is done in 24 bit hardcoded blitting, and was mainly done to see what had to be done. At this point it might be worth noting the work that was done on WME Lite prior to attempting to merge it into the ScummVM-tree as a branch. I started off by forking the original WME-Lite repos onto Github, then I went on to strip out all the dependencies (BASS, Boost, SDL2), as well as removing as much as possible of the "forbidden"-bits (ScummVM doesn't allow direct usage of the C-APIs for file-accesses and random number generation among other things). As I went on, i ported over as much as possible of the ScummVM common code to WME Lite, until I reached the point where I might as well move the WME source over into my ScummVM-fork. 

One of the things I did quite early, was to try to use the ScummVM-image loaders, as opposed to the SDL2-code that was used already in WME Lite, I made that kind of work, but transparency proved problematic for now. I picked the game Dirty Split as a test case for the moment, here is how it looks:

Inventory in the original WME Lite
This is how the inventory in Dirty Split looks in the original prebuilt WME Lite
... and this is how it looks in my modified WME Lite-version, that uses the ScummVM image loader.
Now, for the first ever image of a Wintermute-game running inside ScummVM:

Yup, this is ScummVM, although it isn't really obvious from the headline. Notice how the fence is wrong, and the transparency is off in all sorts of different ways.

The reason for all those interesting artifacts, is mainly that I only added a quick-fix for the colour-keyed transparency, but didn't do anything for the alpha-channel stuff yet. Keen eyes might also notice that the character is a lot bigger in ScummVM, although this is the very same scene, with no movement done yet. This is another case of missing features, right now any bitmap that asks to be drawn, is drawn at it's full size, no matter what it actually says about the wanted size. This gives a few rather odd behaviours at the moment, as for instance, walking away from the screen isn't really obvious:
This is how walking up to the door looks in WME Lite (with a few of my modifications)
... and this is how it looks in ScummVM at the moment.
Clearly, the model's need to scale appropriately, but, atleast input works well enough to get the character to move over there. The action-menu for Dirty Split reveals a few more of the transparency-quirks:

In ScummVM


In WME Lite (with my modifications)
I'll be looking into solving these issues as I implement a more proper drawing-solution in the weeks to come.

Oh well, that's about it for now, I'm off to do my exams, so what better way to end of, than to show how the three variations of the game look when you try to close it? Yes, there are a few minor differencies her, as I didn't go through the menu in ScummVM, but simply Ctrl-C-ed the terminal session.

ScummVM

Modified WME Lite

Original WME Lite

Sunday, 6 May 2012

A beginning

Since I started thinking about approaching this task, I've been doing a little work on the WME Lite code base. I originally forked that repos, and started removing the bits I knew that wouldn't work with ScummVM, which mainly meant stubbing all the dependencies away (Boost, BASS, SDL2), and getting to grips with where STL was used (as ScummVM doesn't allow me to use those).

I then did a bit of "reverse-porting", copying over pieces of ScummVM to that repos, so that I could start replacing pieces of WME-code that depended on things that aren't available in ScummVM, with the proper ScummVM-solutions. This worked up until a certain point, namely getting files to actually load (that would require copying over LARGE amounts of ScummVM-code). At this point I'd gotten to know the WME-codebase a bit better, and also untangled a bit of the header-dependencies that was there (which was usefull, to avoid having to copy the ENTIRE codebase in one go), to the point that I could start copying the files over to a branch in my ScummVM-fork.

This led to a day or so of resolving problems relating to common/forbidden.h (I hadn't purged all the STL/FILE-related pieces of code, since the main reason for moving on from a fork of WME, to a fork of ScummVM, was to get files loading). Additionally I also had to follow the "HOWTO: Engines"  guide in the ScummVM-Wiki, to get the connections correct with the rest of ScummVM

But, in the end, I managed to atleast get a repos that compiles, and links in-tree. Right now, it even detects the game data, and should be just a few hours away from being able to get the scripts out of the data files, to start running the main loop of the engine.

This puts me quite a bit into stuff that was planned a bit further down the road in my GSoC-timeline, but sometimes things tend to happen in rather different orders than what one actually plans, there is still work to be done for what was planned as the first week's work (Refactoring), but happily, I've gotten some help from Sev with automated renaming of variables, which should save me a day or two of manual labour. (Combined with some application of astyle, this atleast gets me of the start line for making the code base follow the ScummVM code-formatting guidelines).

In closing:
I might not be getting too much work into the next few weeks, as I have exams coming up, but I do plan to get the game data files at least loading, and hopefully to start looking through the files for any code formatting-breakages in the upcoming time.

Einar Johan