[ToME 2] Getting serious about the SDL interface (again)

[Therem Harth]

This horse is clearly quite dead. So I think I'll do the obvious thing, and keep beating it until it gets up...

Anyway. ToME 2 currently uses a bunch of disparate interfaces: Win32, X11, GTK2, curses, and SDL. Of those, only GTK2 and SDL are really portable; and GTK2 is obsolete and doesn't like compiling on Windows.

The SDL interface is a lot simpler than Vanilla's. It has no fancy interactive buttons and no mouse support; and for now, that's okay. You have to specify the font from the command prompt, fine; Angband is a geek's game anyway. For now I just want to solve the two main problems...

1. The window is of a static size. For the interface to function properly, the window must change size to suit the size of the font being used. Being able to manuallly resize the window is optional; the size should be gotten right the first time.

2. The interface is extremely slow. Centered view is painful to play with, and uncentered = instadeath from offscreen breathers. Centered view needs to be made usable. I really have no idea why it's so slow though, unless using TrueType fonts just has that much overhead.

Other issues may come up, but those are the only ones for now. I'm working on the first one pretty much as I type this; if anything untoward happens I'll post the code changes. If any of you are willing to share advice... Thanks.

*sigh* Here goes.

[AnonymousHero]

Does SDL support multiple separate windows these days? That's a dealbreaker for me.

If it does, I think I'd actually welcome getting rid of all the other interfaces.

[Therem Harth]

Umm... SDL 1.3 and 2.0 do. SDL 1.2 (which is what almost every distro ships), no.

And this code is an absolute mess. There are two global variables, t_width and t_height, which are supposed to contain (duh?) the width and height of the tiles... But somehow they're getting filled with bogus values somewhere, keeping my "auto-sized" window at a constant (and enormous) size. Feh!

Edit: BTW I'll see about getting your unlua branch compiled and running on 32 bit Linux soonish. Re Windows, that may be a problem; the best I can do right now is a Windows 2000 SP4 virtual machine, and I have no idea how good an indicator that would be of compatibility with XP; let alone 7. Posting here because Gitorious keeps eating my messages.

[AnonymousHero]

@SDL: If the code can be fixed up I certainly won't object to that even if X11 still has to be maintained. (And I suppose it would give us a better shot at supporting Win32/Mac reasonably.)

@Gitorious: Yeah, they seem to be having some weird redirect problem.

[AnonymousHero]

Oh, yeah, and you should probably not do any work based off the "unlua" branch (unless it's actually related to removing Lua code ) -- I may rebase it at some point, so...

The UI stuff is pretty separate from all the Lua bits, so that should present any major problems later.

[Therem Harth]

Thanks. Unfortunately I'm kind of stumped...

Say I want to set the SDL window to a certain size. I can do

arg_width = 640;
arg_height = 480;

And bam, I get a 640x480 window. Cool.

Now say I want the window to be bigger by default if a larger font is selected. In theory, I could do

arg_width = 80 * t_width;
arg_height = 24 * t_height;

And get an 80x24 tile window. But in practice, I invariably get a huge window that takes up most of my screen space. (And is not the 800x600 default, either.) Kind of weird, right? But what's weirder is that

arg_width = 40 * t_width;
arg_height = 12 * t_height;

gives a window of exactly the same size. It's as if multiplying anything by t_width or t_height always gives the exact same nonzero number, which I'm pretty sure should be impossible.

Any clue what might be going on?

[Derakon]

You may be performing an invalid operation; are t_width and t_height defined/initialized at that point? Try printing them out (and printing arg_width and arg_height after you set them) to see what you get.

[Therem Harth]

Thanks.

It looks like t_width and t_height are 0, and SDL is making the window some hard-coded default size. Perhaps the function that sets these variables hasn't run yet.

... Yup, loadAndRenderFont() has not been called yet. Duh! Let's see what I can do about that.

Edit: Got it, bug #1 is fixed. Also, the player can now quit by closing the window without losing any progress in the game.

[Therem Harth]

Okay, I think I see three possibilities for where the performance problem is coming from...

1. The Vanilla Angband SDL interface calls SDL_DisplayFormatAlpha() to convert the display surface to the screen's format. ToME's SDL interface never does that. Doing this conversion on the fly is supposed to be slow. (And no, I don't know what I'm talking about here; I know absolutely nothing about raster graphics.)

2. ToME uses SDL_TTF. Vanilla uses bitmap fonts. I wouldn't think this would make a huge difference on modern machines, however...

3. SDL offers no hardware acceleration on Linux, unless you use fullscreen mode. Otherwise... Zilch. This would make me want to abandon the SDL interface, only it's not really SDL's fault; 2D acceleration on Linux is godawful on most hardware, to the point that unaccelerated graphics perform 2-3 times better. But I suspect that unaccelerated graphics don't cut it for this; judging from the way running in ToME/SDL makes my CPU spike...

(3) obviously can't be solved with SDL, barring the use of OpenGL (in which case you might as well go whole hog and make it pure OpenGL). So before I waste time trying to shoehorn SDL_DisplayFomat() into ToME - can anyone tell me if it would be worth it? Or is it more likely that the TrueType fonts are eating up CPU time?

[Derakon]

I really recommend finding a profiler and using that to pin down where you're spending your time. It's possible that DisplayFormatAlpha is causing you some slowdown, but I wouldn't think any of the things you listed would be gamebreakers. More likely something's just being badly inefficient. It's very easy to write slow code regardless of what libraries you're using, after all.

[AnonymousHero]

I'll second Derakon's advice. Performance problems often turn up in really surprising places, so it's rarely useful to try to guess. (Of course, with experience, you'll start to recognize certain "patterns" as being slow, but that's really something you can only rely on when you have lots of experience... You should also still profile in those cases .)

You'll need to set up profiling in the CMakeLists.txt file. You can find a snippet here (not sure if it'll still work, but...):

http://www.cmake.org/pipermail/cmake/2007-March/013176.html

[Therem Harth]

Thanks. Actually had to add the -pg flag and compile with no optimization (which funnily enough did not worsen the performance).

Anyway the biggest time-eaters are

  %   cumulative   self              self     total           
 time   seconds   seconds    calls   s/call   s/call  name    
 12.58      0.19     0.19      999     0.00     0.00  update_view
 11.92      0.37     0.18    28140     0.00     0.00  process_world
 10.60      0.53     0.16  3052871     0.00     0.00  Rand_div
  5.96      0.62     0.09  1684440     0.00     0.00  perturb_point_end
  5.96      0.71     0.09  1159544     0.00     0.00  Term_queue_char
  5.30      0.79     0.08  1156221     0.00     0.00  map_info
  3.31      0.84     0.05   146608     0.00     0.00  drawTermStuff
  3.31      0.89     0.05   146118     0.00     0.00  Term_text_sdl
  2.65      0.93     0.04  1261836     0.00     0.00  cave_set_feat
  2.65      0.97     0.04   237207     0.00     0.00  luaH_getstr
  2.65      1.01     0.04    36569     0.00     0.00  vprocess_hooks_return
  2.65      1.05     0.04    16673     0.00     0.00  Term_fresh_row_text

update_view and process_world make sense as being inefficient; LoS calculation is probably slow, and process_world is just enormous and handles lots of stuff. Same with generating pseudorandom numbers, from what I've read. drawTermStuff doesn't seem to be that big of a time hog, unless I'm reading things completely wrong.

[AnonymousHero]

Are you running the "maximal stress" scenario, e.g. scrolling with a player-centered view?

Also, I believe the profiler uses sampling, so you'll probably want to run for a little while to get a more reliable picture.

(I'm not entirely sure if the profiler only accounts for the CPU time of the process itself or if "wait time" is counted. I certainly see large CPU spikes for the X11 process when scrolling (when using "compositing") and it would surprise me if this didn't account for the vast majority of the latency observed by the user... but then profiling can lead to surprises)

EDIT: Actually if you add up the UI stuff from that profile, you get quite a considerable proportion of the total time: 12.58% + 5.96% + 3.31% + 3.31% + 2.65% = 28%. It looks like update_view is hideously large... maybe a little refactor of the three steps into individual functions would give a better idea of what, exactly, is so slow in update_view(). I'd expect it to be "Step 3", i.e. the bit where it calls "lite_spot" to actually refresh the UI.

[Therem Harth]

Sorry about the absence, been slightly distracted lately. I'll see what I can do about the SDL code, but don't expect results fast (or necessarily at all).

Also I've been trying to get ToME 2 to compile on Windows 7. It is an epic struggle, and so far one without success; CMake keeps complaining that GCC is not in my PATH, even though it most assuredly is (and the rest of MSys is not). Between that and Git being slow as a beached whale on Windows, I am almost ready to give up.

Not sure what I'd suggest at this point, as ditching the build system entirely seems like a bad idea. Maybe Windows versions should be cross-compiled on Linux and tested under Wine, as originally advised (IIRC by Magnate?). Also, testing the SDL interface under Windows and/or Wine might not be worth the bother.

[AnonymousHero]

Sorry about the absence, been slightly distracted lately. I'll see what I can do about the SDL code, but don't expect results fast (or necessarily at all).

Also I've been trying to get ToME 2 to compile on Windows 7. It is an epic struggle, and so far one without success; CMake keeps complaining that GCC is not in my PATH, even though it most assuredly is (and the rest of MSys is not). Between that and Git being slow as a beached whale on Windows, I am almost ready to give up.

Did you run it exactly like this?

$ cmake -G "MinGW Makefiles"
$ mingw32-make

Not sure what I'd suggest at this point, as ditching the build system entirely seems like a bad idea. Maybe Windows versions should be cross-compiled on Linux and tested under Wine, as originally advised (IIRC by Magnate?). Also, testing the SDL interface under Windows and/or Wine might not be worth the bother.

I have considered "waf" recently while experimenting a little bit with D (which was a pain in CMake), but that requires Python -- though it's less dependent on the user actually having anything else installed.

Suggestions for build systems are most welcome.