Translate Angband 3.1.0 to other language

[buzzkill]

Without going crazy with the source code, you could just create a custom font with a font editor. I'm sure you could find 8 characters that are never used and change them into the ones you need. Probably not the best solution, but certainly doable.

[Nick]

Wow, a lot of information for a newie as me!

Some of us here don't always stick to the topic...

First: what is FA?

FAangband (short for First Age Angband is a variant, which I maintain.

Second: my native language is Spanish, ASCII symbols besides, I just need these eight special symbols:
á, é, í, ó, ú, ń, ˇ, ż

Is necesary to use UTF-8 for handling it? or there are other way?

The other (quicker, but less neat) way of handling it is the extended characters (xchars) which have been mentioned. They are used in some variants, but not in Angband itself.

Third: what object I can start to see in the source code for modify the encoding o Ascci symbols to add this 8 symbols?

You will really need either xchar or UTF-8 support for this. I would recommend that you start by just using the unaccented characters (I'm not sure how badly that effects the meaning in Spanish), and then change over once there is some support for accented characters in Angband.

[ekolis]

As long as the game never refers to years... if you remove the tilde from the ń in ańo, you get abominations like "I have twenty-six anuses" instead of "I am twenty-six years old"!

[nppangband]

Second: my native language is Spanish, ASCII symbols besides, I just need these eight special symbols:
á, é, í, ó, ú, ń, ˇ, ż

Is necesary to use UTF-8 for handling it? or there are other way?

Third: what object I can start to see in the source code for modify the encoding o Ascci symbols to add this 8 symbols?

We are talking about an extended character patch that offers 256 characters instead of the standard ASCII that Angband currently uses. It has the 8 characters you are looking for.

The patch has been long lost, but the code can be found in FAAngband, Sangband and NPPAngband and I am sure a couple others. Just GREP the source for the following: xchar, latin1, my_toupper, my_tolower... and you will find everything you need. THere are about 5-6 functions that need to be added and called in a couple select parts of the source code.

This patch is a workaround. You can't type an accented character directly in a file and have it procesed. Instead you have to put in a special code, like ['e] and it types out é for systems that can handle it, and e for those older systems that can only use ASCII.

By the way, if anyone does have a copy of the extended character patch that we made, I would greatly appreciate a copy. Leon Marrick had it on his server, but that is down. I lost my copy when an older computer I stored it on had a meltdown.

[hernaldo]

Well, i have two options:
Translate FA, I make a couple of tests and it works!, y can put tildes (á) in the text game (for example birth.c), but not in the welcome screen (news.txt), but at least it's a breakthrough.

Or

someone help me to include xchar into Angband, and here i ask:
nppangband remember these functions?

regards!

[nppangband]

Well, i have two options:
Translate FA, I make a couple of tests and it works!, y can put tildes (á) in the text game (for example birth.c), but not in the welcome screen (news.txt), but at least it's a breakthrough.

Or

someone help me to include xchar into Angband, and here i ask:
nppangband remember these functions?

regards!

encode_to_xchar
get_encode
xchar_trans_hook
static const xchar_type latin1_encode
byte char_tables[256][CHAR_TABLE_SLOTS]
xstr_trans
escape_latin1
typedef struct xchar_type xchar_type;
my_toupper (there is a whole series of these in z-form.h...my_tolower, my_isprint, my_isspace are examples. )
T->xchar_hook

You also have to grab the fonts from NPPAngband (in lib/xtra/font), because those have the extended characters in them.

But with all that, I am missing something. Because it only displays the extended characters if a message is formatted instead of printing it directly. For example, in the knowledge screen, where it uses c_put_str, it displays ['e] instead of the accented e. So I am missing a tiny part of the patch, which is why I want to find a copy of the original part of the patch so badly. I will let you know when I find it.

[hernaldo]

Ok thanks!
I'm going to try modify the code. But I've a questions becouse this lines i don't find in the FA or NPPAngband code:

typedef struct xchar_type xchar_type;
T->xchar_hook

what mean "typedef struct xchar_type xchar_type;" and "T->xchar_hook"
are an error?

[nppangband]

Ok thanks!
I'm going to try modify the code. But I've a questions becouse this lines i don't find in the FA or NPPAngband code:



what mean "typedef struct xchar_type xchar_type;" and "T->xchar_hook"
are an error?

You need the structure xchar_type. You can find it (along with many of the functions listed above) in z-form.h, around line 95.

T->xchar_hook needs to be added to the structure term (line 237 of z-term.h). If you grep xchar_hook you will see you need to add some code in main-win.c (and the other operating systems that can handle the extended characters). It is in the function term_data_link:

after these lines:

t->wipe_hook = Term_wipe_win;
t->text_hook = Term_text_win;
t->pict_hook = Term_pict_win;

add this:

t->xchar_hook = Term_xchar_win;

Also, I think I fixed the final problem yesterday...so you won't find this is the source code you downloaded. In z-term.c, you will find a function called term_addstr. Replace that function with the one below. Only the first 6 or 7 lines are different:

errr Term_addstr(int n, byte a, cptr buf)
{
int k;

int w = Term->wid;

errr res = 0;

char s[1024];

/* Copy to a rewriteable string */
my_strcpy(s, buf, 1024);

/* Translate it to 7-bit ASCII or system-specific format */
xstr_trans(s, LATIN1);

/* Handle "unusable" cursor */
if (Term->scr->cu) return (-1);

/* Obtain maximal length */
k = (n < 0) ? (w + 1) : n;

/* Obtain the usable string length */
for (n = 0; (n < k) && s[n]; n++) /* loop */;

/* React to reaching the edge of the screen */
if (Term->scr->cx + n >= w) res = n = w - Term->scr->cx;

/* Queue the first "n" characters for display */
Term_queue_chars(Term->scr->cx, Term->scr->cy, n, a, s);

/* Advance the cursor */
Term->scr->cx += n;

/* Hack -- Notice "Useless" cursor */
if (res) Term->scr->cu = 1;

/* Success (usually) */
return (res);
}

[hernaldo]

OK, it compile!

But, I don't get see the acents (é), maybe I do something wrong.

First: I changed ui-birth.c, the line 135 says:

#define BIRTH_MENU_HELPTEXT \
	"{lightblue}Please select your character...

I put the test code:

#define BIRTH_MENU_HELPTEXT \
	"{lightblue}test á é

Second: I copy the fonts from FA to my Anband / Lib / xtra / font
I think that the text use the "5x8.font", Now it will use "5x8x.font"
Also, I changed the "Makefile" file , because it says:

OBJECTIVE_DATA = \
	copying.txt:${DATA_PATH}/xtra/font  \
	5x8.fon:${DATA_PATH}/xtra/font      \

I changed for:

OBJECTIVE_DATA = \
	copying.txt:${DATA_PATH}/xtra/font  \
	[B]5x8x.fon[/B]:${DATA_PATH}/xtra/font      \

Third: I changed the "config.h" file, because that says:

#define DEFAULT_X11_FONT_0	"10x20"
#define DEFAULT_X11_FONT_1		"9x15"
#define DEFAULT_X11_FONT_2		"9x15"
#define DEFAULT_X11_FONT_3		"5x8"
#define DEFAULT_X11_FONT_4		"5x8"
#define DEFAULT_X11_FONT_5		"5x8"
#define DEFAULT_X11_FONT_6		"5x8"
#define DEFAULT_X11_FONT_7		"5x8"

I changed for:

#define DEFAULT_X11_FONT_0	"10x20"
#define DEFAULT_X11_FONT_1		"9x15"
#define DEFAULT_X11_FONT_2		"9x15"
#define DEFAULT_X11_FONT_3		"5x8x"
#define DEFAULT_X11_FONT_4		"5x8x"
#define DEFAULT_X11_FONT_5		"5x8x"
#define DEFAULT_X11_FONT_6		"5x8x"
#define DEFAULT_X11_FONT_7		"5x8x"

I need to do something to make it work?

thanks and bye!

[nppangband]

You can't enter the characters directly. The game has to be converted from ASCII to unicode for that to be possible. This patch is just a hack to have them display onscreen.

Take a look at the conversion chart in latin1_encode in z-form.c. You have to type "[~n]" in a string and run it through xstr_trans to get the game to type "ń"

[hernaldo]

ok, i understand.
I tried for four hours today to link function with accents xstr_trans. I placed several calls in different parts of the "xstr_trans" function, but nothing worked. I put a MessageBox and I see that I don't get the "encode_to_xchar" function.

I had forgotten to modify the default font in main-win.c, I changed to:

#define DEFAULT_FONT	"8x16x.FON"

Then, I put the [~n]:

#define BIRTH_MENU_HELPTEXT \ 
{lightblue} test [~n]...

Then, the code says (here i don't change nothing):

static void print_menu_instructions(void)
{
	/* Clear screen */
	Term_clear();
	
	/* Output to the screen */
	text_out_hook = text_out_to_screen;
	
	/* Indent output */
	text_out_indent = QUESTION_COL;
	Term_gotoxy(QUESTION_COL, HEADER_ROW);
	
	/* Display some helpful information */
	[B]text_out_e[/B](BIRTH_MENU_HELPTEXT);
	
	/* Reset text_out() indentation */
	text_out_indent = 0;
}

Then, following the code, we have:

void text_out_e(const char *fmt, ...)
{
	
	char buf[1024];
	char smallbuf[1024];
	va_list vp;

	const char *start, *next, *text, *tag;
	size_t textlen, taglen = 0;

	/* Begin the Varargs Stuff */
	va_start(vp, fmt);

	/* Do the va_arg fmt to the buffer */
	(void)[B]vstrnfmt[/B](buf, sizeof(buf), fmt, vp);  -> [B]here I call to xstr_trans function???[/B]

	/* End the Varargs Stuff */
	va_end(vp);

	start = buf;
	while (next_section(start, 0, &text, &textlen, &tag, &taglen, &next))
	{
		int a = -1;

		memcpy(smallbuf, text, textlen);
		smallbuf[textlen] = 0;

		if (tag)
		{
			char tagbuffer[11];

			/* Colour names are less than 11 characters long. */
			assert(taglen < 11);

			memcpy(tagbuffer, tag, taglen);
			tagbuffer[taglen] = '\0';

			a = color_text_to_attr(tagbuffer);
		}
		
		if (a == -1) 
			a = TERM_WHITE;

		
		/* Output now */
		text_out_hook(a, smallbuf);   [B]-> OR HERE I call to xstr_trans function???[/B]


		start = next;
	}
}

This is the code of text_out_hook (i don't think that i might modify):

Extern.h:
extern void (*text_out_hook)(byte a, cptr str);

Variable.c
void (*text_out_hook)(byte a, cptr str);

Finally, it's the code of vstrnfmt functoid in z-form.c, where i see that is ok:

size_t vstrnfmt(char *buf, size_t max, cptr fmt, va_list vp)
{
	cptr s;

	/* The argument is "long" */
	bool do_long;

	/* The argument needs "processing" */
	bool do_xtra;

	/* Bytes used in buffer */
	size_t n;

	/* Bytes used in format sequence */
	size_t q;

	/* Format sequence */
	char aux[128];

	/* Resulting string */
	char tmp[1024];


	/* Fatal error - no buffer length */
	if (!max) quit("Called vstrnfmt() with empty buffer!");


	/* Mega-Hack -- treat "no format" as "empty string" */
	if (!fmt) fmt = "";


	/* Begin the buffer */
	n = 0;

	/* Begin the format string */
	s = fmt;

	/* Scan the format string */
	while (TRUE)
	{
		type_union tval = END;

		/* All done */
		if (!*s) break;

		/* Normal character */
		if (*s != '%')
		{
			/* Check total length */
			if (n == max-1) break;

			/* Save the character */
			buf[n++] = *s++;

			/* Continue */
			continue;
		}

		/* Skip the "percent" */
		s++;

		/* Pre-process "%%" */
		if (*s == '%')
		{
			/* Check total length */
			if (n == max-1) break;

			/* Save the percent */
			buf[n++] = '%';

			/* Skip the "%" */
			s++;

			/* Continue */
			continue;
		}

		/* Pre-process "%n" */
		if (*s == 'n')
		{
			size_t *arg;

			/* Get the next argument */
			arg = va_arg(vp, size_t *);

			/* Save the current length */
			(*arg) = n;

			/* Skip the "n" */
			s++;

			/* Continue */
			continue;
		}


		/* Begin the "aux" string */
		q = 0;

		/* Save the "percent" */
		aux[q++] = '%';

		/* Assume no "long" argument */
		do_long = FALSE;

		/* Assume no "xtra" processing */
		do_xtra = FALSE;

		/* Build the "aux" string */
		while (TRUE)
		{
			/* Error -- format sequence is not terminated */
			if (!*s)
			{
				/* Terminate the buffer */
				buf[0] = '\0';

				/* Return "error" */
				return (0);
			}

			/* Error -- format sequence may be too long */
			if (q > 100)
			{
				/* Terminate the buffer */
				buf[0] = '\0';

				/* Return "error" */
				return (0);
			}

			/* Handle "alphabetic" chars */
			if (isalpha((unsigned char)*s))
			{
				/* Hack -- handle "long" request */
				if (*s == 'l')
				{
					/* Save the character */
					aux[q++] = *s++;

					/* Note the "long" flag */
					do_long = TRUE;
				}

				/* Mega-Hack -- handle "extra-long" request */
				else if (*s == 'L')
				{
					/* Error -- illegal format char */
					buf[0] = '\0';

					/* Return "error" */
					return (0);
				}

				/* Handle normal end of format sequence */
				else
				{
					/* Save the character */
					aux[q++] = *s++;

					/* Stop processing the format sequence */
					break;
				}
			}

			/* Handle "non-alphabetic" chars */
			else
			{
				/* Hack -- Handle 'star' (for "variable length" argument) */
				if (*s == '*')
				{
					int arg;

					/* Get the next argument */
					arg = va_arg(vp, int);

					/* Hack -- append the "length" */
					sprintf(aux + q, "%d", arg);

					/* Hack -- accept the "length" */
					while (aux[q]) q++;

					/* Skip the "*" */
					s++;
				}

				/* Mega-Hack -- Handle 'caret' (for "uppercase" request) */
				else if (*s == '^')
				{
					/* Note the "xtra" flag */
					do_xtra = TRUE;

					/* Skip the "^" */
					s++;
				}

				/* Collect "normal" characters (digits, "-", "+", ".", etc) */
				else
				{
					/* Save the character */
					aux[q++] = *s++;
				}
			}
		}


		/* Terminate "aux" */
		aux[q] = '\0';

		/* Clear "tmp" */
		tmp[0] = '\0';

		/* Parse a type_union */
		if (aux[q-1] == 'y')
		{
			tval = va_arg(vp, type_union);

			if (do_long)
			{
				/* Error -- illegal type_union argument */
				buf[0] = '\0';

				/* Return "error" */
				return (0);
			}

			/* Replace aux terminator with proper printf char */
			if (tval.t == T_CHAR) aux[q-1] = 'c';
			else if (tval.t == T_INT) aux[q-1] = 'd';
			else if (tval.t == T_FLOAT) aux[q-1] = 'f';
			else if (tval.t == T_STRING) aux[q-1] = 's';
			else
			{ 
				buf[0] = '\0';
				return (0);
			}
		}

		/* Process the "format" symbol */
		switch (aux[q-1])
		{
			/* Simple Character -- standard format */
			case 'c':
			{
				int arg;

				/* Get the next argument */
				arg = tval.t == T_END ? va_arg(vp, int) : tval.u.c;

				/* Format the argument */
				sprintf(tmp, aux, arg);

				/* Done */
				break;
			}

			/* Signed Integers -- standard format */
			case 'd': case 'i':
			{
				if (do_long)
				{
					long arg;

					/* Get the next argument */
					arg = va_arg(vp, long);

					/* Format the argument */
					sprintf(tmp, aux, arg);
				}
				else
				{
					int arg;

					/* Get the next argument */
					arg = tval.t == T_END ? va_arg(vp, int) : tval.u.i;

					/* Format the argument */
					sprintf(tmp, aux, arg);
				}

				/* Done */
				break;
			}

			/* Unsigned Integers -- various formats */
			case 'u': case 'o': case 'x': case 'X':
			{
				if (do_long)
				{
					unsigned long arg;

					/* Get the next argument */
					arg = va_arg(vp, unsigned long);

					/* Format the argument */
					sprintf(tmp, aux, arg);
				}
				else
				{
					unsigned int arg;

					/* Get the next argument */
					arg = va_arg(vp, unsigned int);

					/* Format the argument */
					sprintf(tmp, aux, arg);
				}

				/* Done */
				break;
			}

			/* Floating Point -- various formats */
			case 'f':
			case 'e': case 'E':
			case 'g': case 'G':
			{
				double arg;

				/* Get the next argument */
				arg = tval.t == T_END ? va_arg(vp, double) : tval.u.f;

				/* Format the argument */
				sprintf(tmp, aux, arg);

				/* Done */
				break;
			}

			/* Pointer -- implementation varies */
			case 'p':
			{
				void *arg;

				/* Get the next argument */
				arg = va_arg(vp, void*);

				/* Format the argument */
				sprintf(tmp, aux, arg);

				/* Done */
				break;
			}

			/* String */
			case 's':
			{
				cptr arg;
				char arg2[1024];

				/* Get the next argument */
				arg = tval.t == T_END ? va_arg(vp, cptr) : tval.u.s;

				/* Hack -- convert NULL to EMPTY */
				if (!arg) arg = "";

				/* Prevent buffer overflows */
				(void)my_strcpy(arg2, arg, sizeof(arg2));

				/* Translate it to 8-bit (Latin-1) */
 				xstr_trans(arg2, LATIN1);

				/* Format the argument */
				sprintf(tmp, aux, arg2);

				/* Done */
				break;
			}

#if 0 /* Later */
			/* Binary */
			case 'b':
			{
				int arg;
				size_t i, max = 32;
				u32b bitmask;
				char out[32 + 1];

				/* Get the next argument */
				arg = va_arg(vp, int);

				/* Check our aux string */
				switch (aux[0])
				{
					case '1': max = 2;  break;
					case '2': max = 4;  break;
					case '3': max = 8;  break;
					case '4': max = 16; break;
					default: 
					case '5': max = 32; break;
				}
				/* Format specially */
				for (i = 1; i <= max; i++, bitmask *= 2)
				{
					if (arg & bitmask) out[max - i] = '1';
					else out[max - i] = '0';
				}

				/* Terminate */
				out[max] = '\0';

				/* Append the argument */
				my_strcpy(tmp, out, sizeof tmp);

				/* Done */
				break;
			}
#endif

			/* Oops */
			default:
			{
				/* Error -- illegal format char */
				buf[0] = '\0';

				/* Return "error" */
				return (0);
			}
		}


		/* Mega-Hack -- handle "capitalization" */
		if (do_xtra)
		{
			for (q = 0; tmp[q]; q++)
			{
				/* Notice first non-space */
				if (!my_isspace((unsigned char)tmp[q]))
				{
					/* Capitalize if possible */
					if (my_islower((unsigned char)tmp[q]))
						tmp[q] = my_toupper((unsigned char)tmp[q]);

					/* Done */
					break;
				}
			}
		}

		/* Now append "tmp" to "buf" */
		for (q = 0; tmp[q]; q++)
		{
			/* Check total length */
			if (n == max-1) break;

			/* Save the character */
			buf[n++] = tmp[q];
		}
	}


	/* Terminate buffer */
	buf[n] = '\0';

	/* Return length */
	return (n);
}

What else I can check?

thanks a lot!!

[nppangband]

ok, i understand.
I tried for four hours today to link function with accents xstr_trans. I placed several calls in different parts of the "xstr_trans" function, but nothing worked. I put a MessageBox and I see that I don't get the "encode_to_xchar" function.

{snip}

What else I can check?

thanks a lot!!

THere is no need to edit text_out_e. It already calls vstrnfmt, which handles the conversation. You can paste the vstrnfmt directly from NPP into Angband and that works.

Which operating system are you using to test this?

[hernaldo]

I using Windows XP Prof, i compile using Mingw from the command line
> mingw32-make.exe -f makefile.win

I checked all and see that it works (finally, yes!) for the dinamic as "Choose New or Open from file Menu" o "STR, Dex points" or "Your 'race' determines various intrinsic factors and bonuses."

But for the static text as "Bug reports to ..." or "For resourses and link..." or "Please select your character from the menu below", don't' work yet.
It is assumed that when any text is show in the screen, it reaching "vstrnfmt" and there It get to "s" case:

	
/* String */
case 's':
{
	cptr arg;
	char arg2[1024];

	/* Get the next argument */
	arg = va_arg(vp, cptr);

	/* Hack -- convert NULL to EMPTY */
	if (!arg) arg = "";

	/* Prevent buffer overflows */
	(void)my_strcpy(arg2, arg, sizeof(arg2));

	/* Translate it to 8-bit (Latin-1) */
	[B]xstr_trans[/B](arg2, LATIN1);
        ...

But when I put a message box, only the dinamic text pass for here, but the static text does'nt!!
-I shall not want to copy another function?


Besides, is that I can't use the Latin character ż=191 because it shows "?" in the game. In the "static const xchar_type latin1_encode[]" table does not appear because the table start in the latin character 192, but it appear in "byte char_tables[256][CHAR_TABLE_SLOTS]" table.
-that could be happening? I need add this character to the table? or add it in the fonts?

thanks for your support!

[nppangband]

I tried your test baove, but I couldn't get it to work either. It only seems to work if it is read from the edit files (look in artifact.txt, or monster.txt for examples), which, up until now, is all we have needed it to work.

A simple one to test is in p-class.txt. Change the entry for Dunadan and change it to:

D['u]nadan

and then start up a new character and see if it displays correctly. That should work as a starting point.

[takkaria]

I'm inclined to favour the "include xchars as a quick fix, but aim to move to UTF-8 asap" approach, but I agree we need takkaria's approval on the first bit of that. From my personal point of view, I would like to add xchars to the AngbandBase code (as I'm planning to incorporate that into FA, and I don't want to remove xchars from FA); however I want AngbandBase to remain aligned with V. So I'm going to do unconnected stuff (or possibly nothing ) on AngbandBase until the situation clarifies.

As to unicode handling - I'm guessing pretty much everything can deal with it, but only guessing.

Moving to unicode requires a change of fonts aside from all the code changes, so it will be a real PITA and it's possible the benefits will not outweigh the costs. So, I guess including xchars in V in the meantime is a better idea than blocking it.

PS. translation is something i've always wanted to support in the game, but I think to do it properly you do really need to have a scripting language to hold the language packs, given the complexity of angband's text handling. maybe we should be bringing lua back...