Reply to topic  [ 8 posts ] 
yexec_include "1." 
Author Message
Yorick Master

Joined: Tue Mar 07, 2006 10:31 pm
Posts: 125
Location: Meudon, France
Post yexec_include "1."
Hi,

I'm trying to use yexec_include() to interpret arbitrary code.

In the interpreted version, this yields an expected error:
Code:
include, "1.";
because include() recognizes a file name due to the dot. This is fine. This can be avoided by passing an array of strings or an array of chars to include:
Code:
include, ["1."];
include, ['1', '.'];

I would expect the compiled version to behave the same. I expect an error with that:
Code:
string_t * code = ypush_q(0);
*code=p_strcpy("1.");
yexec_include(0,1);
and indeed:
Code:
ERROR (*main*) missing include file specified in include function
WARNING detailed line number information unavailable
now at pc= 8 (of 15), failed at pc= 10
  LINE: 6  FILE: /home/thibaut/git/yorick-gtk/yorick_gtk.i

I expect no error for the two following codes::
Code:
long dims[Y_DIMSIZE]={1,1};
string_t * code = ypush_q(dims);
code[0]=p_strcpy("1.");
yexec_include(0,1);
Code:
long dims[Y_DIMSIZE]={1,3};
char * code = ypush_c(dims);
strcpy(code, "1.");
yexec_include(0,1);
But actually there is one:
Code:
SYNTAX: syntax error near <EOF>
  LINE: 1  FILE: (vopen file)
ERROR (*main*) ****ABORTING PARSE**** too many errors
WARNING detailed line number information unavailable
now at pc= 8 (of 15), failed at pc= 10
  LINE: 1  FILE: (vopen file)
I have no problem if there is no dot in my input line. Seeing how yexec_include() itself calls Y_include(), I can't make any sense of that. Who's buggy here, Yorick or I?


Tue Apr 09, 2013 8:26 am
Profile WWW
Yorick Master

Joined: Tue Mar 07, 2006 10:31 pm
Posts: 125
Location: Meudon, France
Post [RESOLVED] yexec_include "1."
Hi,

I've found the root of my problem: Gtk.init() installs my locale (French), where the decimal separator is a coma, not a dot. Actually, even at the prompt:
Code:
thibaut@b-wing:~/git/gy$ rlwrap yorick
Copyright (c) 2005.  The Regents of the University of California.
All rights reserved.  Yorick 2.2.02 ready.  For help type 'help'
> #include "gy.i"
> Gtk=gy_require("Gtk");
> Gtk.init(0,);
[]
> 1.0
SYNTAX: syntax error near .0
SYNTAX: syntax error near <EOF>
> pi
3,14159


It would not make any sense to "fix" Yorick to use a coma instead of a dot in France (and it would be very difficult). I'll try fixing my plug-in to not install the locale. export LC_ALL=C does it in the meantime.

Sorry for the noise.

Regards, Thibaut.


Thu Apr 11, 2013 11:25 am
Profile WWW
Yorick Master

Joined: Mon Nov 22, 2004 9:43 am
Posts: 354
Location: Livermore, CA, USA
Post Re: yexec_include "1."
Yes. I made a concious decision to stick with the C locale in yorick. If it makes you feel any better, I've had trouble with the English locale as well -- it changes the order of alphabetic sort functions, even if . remains the decimal point (at least in the US variant). The locales are a pain in the butt for simple C programs as well as yorick. I think it was a big mistake to try to get UNIX shell commands to understand liguistic differences; that is much better left to a higher level since nobody cares about making low level stuff pretty nearly as much as they care about making it correct and robust.

That said, GTK+ is of course a high level interface, which must and should care about locale. We'll need to figure out a way for yorick's C library calls (basically sscanf and maybe some sorting functions) to use C locale no matter wht the environment variable is set to. I don't know how to do that off the top of my head, but without it, there's no way to guarantee that yorick programs work the same for everybody.


Fri May 17, 2013 8:43 pm
Profile
Yorick Master

Joined: Tue Mar 07, 2006 10:31 pm
Posts: 125
Location: Meudon, France
Post Re: yexec_include "1."
Dear Dave,

Stackoverflow has a question about that:
http://stackoverflow.com/questions/1391 ... -like-3-14

From what I understand, you or me could surround calls to locale-dependent functions (either in Yorick or in the Gtk plug-in) to calls to the uselocale family of functions. This is new in POSIX.1-2008.

This should be doable a bit like this in Yorick (with a compile-time conditional on uselocale availability):

Code:
// upon Yorick initialization
setlocale(LC_ALL, "C");
locale_t c_locale = locale_t duplocale(LC_GLOBAL_LOCALE);
...
locale_t prev = duplocale(LC_GLOBAL_LOCALE);
uselocale(c_locale);
sscanf(...);
uselocate(prev);


The reverse could be done in my plug-in instead, I guess. My worry is to be able to reset the locale to "C" in any error condition (such as user interrupt etc.).

There is a family of C functions which take a locate_t object as an argument, but I could not make out yet which ones are POSIX by now: http://www.manpagez.com/man/3/xlocale/

Regards, Thibaut.


Tue May 21, 2013 2:01 am
Profile WWW
Yorick Master

Joined: Mon Nov 22, 2004 9:43 am
Posts: 354
Location: Livermore, CA, USA
Post Re: yexec_include "1."
For the purposes of a Debian yorick package, the correct thing to do, in my opinion, is to make /usr/bin/yorick a script which sets the locale to C, then execs the yorick binary. That strategy works in any POSIX OS no matter how old. Having yorick be a script provides a lot of flexibility for coping with this kind of annoyance it the future. Many other packages do this (e.g.- firefox), so there's lots of precedent.


Thu May 30, 2013 12:44 pm
Profile
Yorick Master

Joined: Tue Mar 07, 2006 10:31 pm
Posts: 125
Location: Meudon, France
Post Re: yexec_include "1."
Dear Dave,

This does not solve the problem at hand, namely having portions of compiled codes work in a localized environment while the Yorick parser itself sticks to the C locale...

Kind regards, Thibaut.


Thu Jun 06, 2013 8:04 am
Profile WWW
Yorick Master

Joined: Mon Nov 22, 2004 9:43 am
Posts: 354
Location: Livermore, CA, USA
Post Re: yexec_include "1."
Refer to section 7.11.1.1 of the ISO/IEC 9899:TC3 (WG14/N1256) (C99 standard) or ISO/IEC 9899:2011 (N1570) (C11 standard), the C library standards. (The open-std.org website points to public versions of many important programming standards, e.g.- http://www.open-std.org/jtc1/sc22/wg14/www/standards for C language standards. This one has been stable for many years.) Paragraph 4 reads:
Quote:
At program startup, the equivalent of
setlocale(LC_ALL, "C");
is executed.

Anyway, I retract what I said about making /usr/bin/yorick a shell script for this reason. It might be a good idea for other reasons, but using it to set LANG is not going to accomplish anything useful. Because libc is written to make sscanf, strcmp, and other functions used throughout yorick depend on something other than their arguments, it will always be possible to write a plugin which changes their behavior and breaks yorick in the manner you describe. Messing with setlocale is just something that plugins cannot do -- or if they do, they are responsible for resetting everything to C locale before returning control to the interpreter. The C standard already guarantees that unless you mess with the locale intentionally (which yorick doesn't), you will get C locale behavior, which yorick relies upon.

I hadn't appreciated that GTK calls setlocale. It's going to break most interpreted languages. You might check to see what the various python GTK packages do about it. It's possible that they've written the python internals like the parser to not call the libc string.h functions, but it seems unlikely. You have to decide whether you really need the LANG locale for your GTK windows. If not, surely GTK (or, I suspect pango) offers some way to set a locale other than with the LANG environment variable? Ah, yes, see: https://developer.gnome.org/gtk3/stable ... -setlocale . I notice that Ruby has a similar problem, as well as the GTK API for fortran, and probably many other places.


Sat Jun 08, 2013 8:16 am
Profile
Yorick Master

Joined: Tue Mar 07, 2006 10:31 pm
Posts: 125
Location: Meudon, France
Post Re: yexec_include "1."
Dear Dave,

What I do already is call setlocale right after gtk_init to reset LC_NUMERIC to C to avoid the breakage I have noticed. I also provide and interpreted interface to setlocale so that the end user can decide which part of the locale to set (but I force LC_NUMERIC to C). Therefore, the problem at hand is solved.

However:
  • I'm not sure that the only problem is with LC_NUMERIC and I would prefer to fix the problems rather than let the user do it;
  • resetting everything to C is unsatisfactory as localization of GUIs is a worthy goal.
The uselocale family of functions look interesting to get more freedom and more reliability at the same time. In particular, they can be used to set a different locale in each thread.

Having said that, this issue is not urgent. All those interfaces we are talking about are for scientific software and few will be localized properly anyway. Currently I'm reflecting on how to get Yorick to draw on a Gtk widget (or, perhaps, cairo surface?) rather than calling xlib directly, in order to make it possible to design cross-platform GUIs embedding a Yorick window.


Wed Jun 12, 2013 2:07 am
Profile WWW
Display posts from previous:  Sort by  
Reply to topic   [ 8 posts ] 

Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by STSoftware for PTF.