Reply to topic  [ 5 posts ] 
Using yorick-doc without git 
Author Message
Yorick Master

Joined: Wed Jun 01, 2005 11:34 am
Posts: 112
Post Using yorick-doc without git
I thought I'd try using the newly released yorick-doc to generate HTML for our local code. We keep our code in a Mercurial repository instead of a git repository. I got a bunch of warnings when I ran the first pass, and then the second pass didn't seem to do anything.

So, I did a simpler case for testing: a single file in a directory, without any version control. The file is /home/dnagle/test/test.i and it contains:

Code:
local foo;
/* DOCUMENT foo
    Foo is now documented.
    SEE ALSO: bar
*/

func bar {
/* DOCUMENT bar
    Bar is now documented.
    SEE ALSO: foo
*/
}


I cloned yorick-doc to /home/dnagle/yorick-doc. Within that directory, I ran "perl ydoc.pl /home/dnagle/test". I received the following warnings on the console:

Code:
Can't open gh-pages/_config.yml: No such file or directory at ydoc.pl line 33.
Use of uninitialized value in pattern match (m//) at ydoc.pl line 137.
Use of uninitialized value in pattern match (m//) at ydoc.pl line 138.
Use of uninitialized value in pattern match (m//) at ydoc.pl line 139.
Use of uninitialized value in concatenation (.) or string at ydoc.pl line 489, <> line 1.
Use of uninitialized value in string eq at ydoc.pl line 66, <> line 1.


Despite the warning, I do now indeed have a directory /home/dnagle/yorick-doc/gh-pages that contains a single file "index.html" that contains:

Code:
---
layout: default
headline:
---
<p class="sectop">Package test (in test.i) - </p>
<p class="sectop">Index of documented functions or symbols:</p>
<div class="ndex0s">
  <div class="ndex1">
    <h3>B</h3>
    <p><a href="index.html#bar">bar</a></p>
  </div>
  <div class="ndex2">
    <h2>F</h3>
  </div>
  <div class="ndex3">
    <p><a href="index.html#foo">foo</a></p>
  </div>
</div>
<div class="docblock"><a name="bar"></a><h3>bar</h3>
  <pre>DOCUMENT bar
Bar is now documented.</pre>
  <p>SEE ALSO: <a href="index.html#foo">foo</a></p>
</div>
<div class="docblock"><a name="foo"></a><h3>foo</h3>
  <pre>DOCUMENT foo
Foo is now documented.</pre>
  <p>SEE ALSO: <a href="index.html#bar">bar</a></p>
</div>


I then tried running "perl ydoc.pl" to generate the HTML and received the following warning messages:
Code:
Can't open gh-pages/_config.yml: No such file or directory at ydoc.pl line 33.
Use of uninitialized value in pattern match (m//) at ydoc.pl line 711, <> line 1.
Can't open /home/dnagle/yorick-doc/gh-pages/_site/../_layouts/default.html: No such file or directory at ydoc.pl line 33, <> chunk 1.
Use of uninitialized value in pattern match (m//) at ydoc.pl line 769, <> line 1.
Use of uninitialized value in substitution (s///) at ydoc.pl line 773, <> line 1.
Use of uninitialized value in substitution (s///) at ydoc.pl line 775, <> line 1.
Use of uninitialized value in substitution (s///) at ydoc.pl line 776, <> line 1.
Use of uninitialized value in print at ydoc.pl line 34, <> line 1.


After running this, I now have a new file /home/dnagle/yorick-doc/gh-pages/_site/index.html. However, this file is empty.

So am I using this as intended? That is, is it supposed to work with code that's not in a git repository? If it should be working, any thoughts on why it's not?


As an aside, instead of using HTML syntax such as:
Code:
<a name="foo"></a><h3>foo</h3>

You might instead consider this:
Code:
<h3 id="foo">foo</h3>

Some discussion on name versus id can be found at http://stackoverflow.com/questions/4847 ... name-or-id and http://stackoverflow.com/questions/1397 ... butes-html


Wed Nov 02, 2011 8:10 am
Profile
Yorick Master

Joined: Mon Nov 22, 2004 9:43 am
Posts: 354
Location: Livermore, CA, USA
Post Re: Using yorick-doc without git
Uh oh. I set up your test, and it works fine for me, so I don't have any way to debug it at this end...

I designed yorick-doc to work fine whether or not you have git on your machine, although of course the -u option won't work. All you need is a functioning perl. I even built in a partial jekyll emulator so you don't need ruby to build the actual pages. So the answer is, I'm trying hard to make this work with the only prerequisite being perl. If you are trying to maintain a public webpage, you will be somewhat crippled by the fact that yorick-doc can only get any customizations you have done from a gh-pages branch in a git repo. The most serious problem is that you won't be able to write a custom _includes/package.html file describing your package, or rather, you'll have to insert that yourself before running the jekyll emulator every time you rebuild. Your options are to script that part yourself (it would be easy in a Makefile), or to relent and make your yorick plugin directory a git repo. My understanding is, that Mercurial can act as an excellent front-end for git repositories (unlike the other way round), so that you never need to use the git front-end, so this may be less painful for you than it sounds. In the longer term, I should probably just add a few more command line switches to ydoc.pl to allow you to specify a default package.html file and several other common customization options directly.

The problem appears to be at line 104 of ydoc.pl. Either your git is returning success, falsely asserting that your test/ directory is a git repository (that is, that test/.git exists), or perl is not reporting the correct status, or your OS (or tar) doesn't propagate the failure status properly. Try the following:

Change line 104 of ydoc.pl to this:
Code:
            system "git archive --remote=../$srcpath $nm | tar xf -";
print "nm=$nm   status=$?\n";

and call the resulting perl script junk.pl. This will let use see the stderr from the git and tar processes (and stdout from tar), then print the status each time. Evidently, on your machine it will return status=0, indicating that it found and extracted a gh-pages branch from the git repo in your test/ directory! If so, we'll have to figure out some way to work around this problem on your (broken?) platform.

Here's the output on my Ubuntu 11.10 machine:
Code:
dave@dora:~/gh/yorick-doc$ ls -Rla ../test
../test:
total 12
drwxrwxr-x  2 dave dave 4096 2011-11-03 13:06 .
drwxr-xr-x 32 dave dave 4096 2011-11-03 13:04 ..
-rw-rw-r--  1 dave dave  153 2011-11-03 13:06 test.i
dave@dora:~/gh/yorick-doc$ git --version
git version 1.7.5.4
dave@dora:~/gh/yorick-doc$ perl -w junk.pl ../test
remote: fatal: '../../test' does not appear to be a git repository       
remote: git upload-archive: archiver died with error
fatal: sent error to the client: git upload-archive: archiver died with error
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
nm=gh-pages   status=512
remote: fatal: '../../test' does not appear to be a git repository       
fatal: sent error to the client: git upload-archive: archiver died with error
remote: git upload-archive: archiver died with error
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
nm=origin/gh-pages   status=512
dave@dora:~/gh/yorick-doc$


The errors indicate correct behavior. Because of the non-zero status they generate, the code never reaches line 137, which is where it comes off the track in your case. Note the -w on my perl execute line. This may cause some more perl warning messages which might help us figure out what is going on. I'm a beginner at perl programming; I'm using v5.12.4 if that makes a difference.

You might also check that you have the latest and greatest yorick-doc source. For about a day an early dead-on-arrival version was on github, while I was scrambling to fix the problems I noticed only after I made the public release. With any luck you just got the original broken version. The git log command should tell you you are at commit 55eeb78fb3.


Thu Nov 03, 2011 12:56 pm
Profile
Yorick Master

Joined: Wed Jun 01, 2005 11:34 am
Posts: 112
Post Re: Using yorick-doc without git
I tried it on another machine and it worked!

Here's some output from the machine that fails:

Code:
$ perl -w junk.pl ../test
nm=gh-pages  status=0
Can't open gh-pages/_config.yml: No such file or directory at ydoc.pl line 33.
Use of uninitialized value in pattern match (m//) at ydoc.pl line 138.
Use of uninitialized value in pattern match (m//) at ydoc.pl line 139.
Use of uninitialized value in pattern match (m//) at ydoc.pl line 140.
Use of uninitialized value in concatenation (.) or string at ydoc.pl line 490, <> line 1.
Use of uninitialized value in string eq at ydoc.pl line 66, <> line 1.
$ git archive --remote=../test gh-pages | tar xf -
remote: fatal: '../test' does not appear to be a git repository
fatal: sent error to client: git upload-archive: archiver died with error
remote: git upload-archive: archiver died with error
$ echo $?
0
$ git archive --remote=../test gh-pages > /dev/null
remote: fatal: '../test' does not appear to be a git repository
fatal: sent error to client: git upload-archive: archiver died with error
remote: git upload-archive: archiver died with error
$ echo $?
1
$ uname -a
Linux automatix.er.usgs.gov 2.6.18-164.11.1.el5 #1 SMP Wed Jan 6 13:26:04 EST 2010 x86_64 x86_64 x86_64 GNU/Linux
$ git --version
git version 1.7.4.1
$ tar --version | head -n 1
tar (GNU tar) 1.15.1


And here's some output from the machine that succeeds:

Code:
$ git archive --remote=../test gh-pages | tar xf -
remote: fatal: '../test' does not appear to be a git repository
remote: git upload-archive: archiver died with error
fatal: sent error to client: git upload-archive: archiver died with error
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
$ echo $?
2
$ git archive --remote=../test gh-pages > /dev/null
remote: fatal: '../test' does not appear to be a git repository
remote: git upload-archive: archiver died with error
fatal: sent error to client: git upload-archive: archiver died with error
$ echo $?
1
$ uname -a
Linux asterix.er.usgs.gov 2.6.35.13-92.fc14.x86_64 #1 SMP Sat May 21 17:26:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
$ git --version
git version 1.7.4.4
$ tar --version | head -n 1
tar (GNU tar) 1.23


Taking at look at the GNU tar release history, I think I discovered the issue here. The notes for version 1.19 (released 2007-10-10) includes:

Quote:
Recognition of broken archives.
When supplied an archive smaller than 512 bytes in reading mode (-x, -t), previous tar versions silently ignored it, exiting with code 0. It is fixed. Tar now issues the following diagnostic message: This does not look like a tar archive, and exits with code 2.


So this is a known issue for the version of tar I'm using, unfortunately. Thanks for your help in tracking the problem's source down.


Fri Nov 04, 2011 6:50 am
Profile
Yorick Master

Joined: Mon Nov 22, 2004 9:43 am
Posts: 354
Location: Livermore, CA, USA
Post Re: Using yorick-doc without git
Okay, I added a workaround for this GNU tar bug (instead of checking exit status, I check if the required _layouts directory has actually been copied). It should work now on your "problem platform" -- let me know if not.

I also added a -d ghpath option to permit you to store customizations in your file system instead of in a gh-pages git branch. That should make it much easier to be entirely cut free from git, wihtout losing any of the yorick-doc functionality.

Thanks very much for pointing out this problem.


Sat Nov 05, 2011 4:15 pm
Profile
Yorick Master

Joined: Mon Nov 22, 2004 9:43 am
Posts: 354
Location: Livermore, CA, USA
Post Re: Using yorick-doc without git
I fumbled the git merge of the gh-pages to master branch, which would have led to obscure results. Fixed as of 12/Nov.


Sat Nov 12, 2011 10:02 am
Profile
Display posts from previous:  Sort by  
Reply to topic   [ 5 posts ] 

Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by STSoftware for PTF.