Reply to topic  [ 2 posts ] 
rdline error with long lines 
Author Message
Yorick Master

Joined: Wed Jun 01, 2005 11:34 am
Posts: 112
Post rdline error with long lines
I've encountered the following unexpected behavior. If a text file contains any line that contains 16,384 or more characters, attempting to read that line in yorick with rdline or rdfile will result in the following error:

ERROR (rdfile) ****ABORTING READ**** input file not ASCII text

I looked through the Yorick code and found the error message is only generated from one spot, function NextInLine in ascio.c. The code indicates that this error should be generated when no EOF is encountered.

Since 16,384 is 2^14, I'm guessing that means that the buffer it uses to read in data is 2^14 in size. I'm not sure if the buffer is system dependent or not; if it is, then the line-length limit will probably vary by system as well.

I didn't bother hunting through the code any further to figure out exactly why this is giving rise to an error, since knowing the problem allowed me to put a work-around in place. But it does seem like errant behavior?

I tested this on both yorick 2.1.05 (under 64bit Linux) and yorick 2.1.04 (under 32bit Linux).


Wed Aug 27, 2008 1:56 pm
Profile
Yorick Master

Joined: Mon Nov 22, 2004 9:43 am
Posts: 354
Location: Livermore, CA, USA
Post 
That's not a bug, it's a feature.

The 16384 limit is actually in yio.c (browse to [yorick] / yorick / yorick / yio.c in CVS), in the function Ygets. This is actually reached only after the line buffer has been doubled several times -- eventually, you have to decide that the line will never end and call it off.

The only place I've hit this limit is in trying to read pseudo-text files (like XML files) that are designed to be manipulated only by programs, not by text editors. Many utilities, not only yorick, will have trouble with this sort of pseudo-text. The truth is, these are binary files, and you need to treat them as such. Yorick can read them by using open(filename,"rb"), then using the _read function to grab the contents in a char array. You can convert a char array x to a string with string(&x), with no limits on length.


Wed Aug 27, 2008 7:37 pm
Profile
Display posts from previous:  Sort by  
Reply to topic   [ 2 posts ] 

Who is online

Users browsing this forum: Yahoo [Bot] and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by STSoftware for PTF.