Reply to topic  [ 4 posts ] 
Circular includes 
Author Message
Yorick Master

Joined: Wed Jun 01, 2005 11:34 am
Posts: 112
Post Circular includes
Suppose I have these two files:

file1.i contains:
Code:
write, "file1";
require, "file2.i";


file2.i contains:
Code:
write, "file2";
require, "file1.i";


Then, in Yorick, this happens:

Code:
> require, "file1.i"
file1
file2
file1
file2
(trimmed many many lines for brevity)
file1
file2
file1
ERROR (*main*) missing include file specified in require function
WARNING detailed number information unavailable
now at pc= 1 (of 11), failed at pc= 5
  LINE: 2  FILE: /home/dnagle/file1.i
To enter debug mode, type <RETURN> now (then dbexit to get out)
>


Obviously, what's happening here is that it is infinitely recursing until it hits its limit and then gives up. So I understand why it happens. A lot of our codebase has a lot of mutual requires, so I solved the issue by having a file like "master.i" with contents like:

Code:
if(is_void(__master_includes_included)) {
  __master_includes_included = 1;
  require, "file1.i";
  require, "file2.i";
}


Then, at the top of file1.i and file2.i, I just require master.i. This works well enough, but it has two undesirable side effects:
1. We're unable to selectively include a subset of our files. It'd be too complex to have a separate master file for each sub-group of mutual includes, so they're all in one.
2. We're no longer documenting which specific files we actually have dependencies on.

Now, neither of those side effects are the end of the world. But I'm wondering, is there a better way to handle this? And just to confirm... the above is the expected behavior, correct?


Thu Dec 08, 2011 12:50 pm
Profile
Yorick Master

Joined: Mon Nov 22, 2004 9:43 am
Posts: 354
Location: Livermore, CA, USA
Post Re: Circular includes
I do not understand why you ever want to do this. The idea of require is when you need a function func2 defined in file2 in order to process file1, you require file2 at the top of file1. Whatever capability file2 provides, I don't see how it can depend on a capability file1 provides, if file1 requires the capability file2 provides...

Leaving that aside, let's suppose you have two functions func1 and func2 which call each other recursively, and let's suppose that (rather illogically), you put them in different files file1 and file2, respectively. While it is true that you can't put a require at the top of each file, you can put a require inside of each function. In general, it makes sense to put the require inside a function, whenever that function is the only place you need the required capability, and you will often include the file but never call that function. For the same reason, you can even put the require inside a rarely used branch of a function.

If you really want to put the mutual requires outside of all functions, all you need do is implement a "provide" function to halt the recursion. Since you are bypassing yorick's non-recursive require by doing that, you may as well create your own require function. Something like this would do the trick:
Code:
func rrequire(feature,fname)
{
  if (!feature) include,fname,1;
}

Then at the top of each of your files, you put this:
Code:
my_feature = 1;
rrequire, other_feature, "other-file";


But I still don't understand why you would ever want to do this... Surely it is really a dependency tree, and you're just trying to make sure only the minimum sub-tree is included? But then, why wouldn't you simply put the requires for each child file at the top of that file?


Thu Dec 08, 2011 2:12 pm
Profile
Yorick Master

Joined: Wed Jun 01, 2005 11:34 am
Posts: 112
Post Re: Circular includes
We have a codebase of 129 Yorick files developed over the last decade. The dependencies are a bit like spaghetti at times and we do in fact have files that are mutually dependent, or worse, dependent in a convoluted loop. Should they be? Probably not. But unfortunately, they are.

Thanks for your suggestions. I want to try to move away from the current framework (having a master include file) back into having each file explicitly list its dependencies. Using something like "rrequire" will let me do that, and then later maybe I can try to untangle some of the mutual dependencies and loops.


Wed Dec 14, 2011 10:44 am
Profile
Yorick Master

Joined: Wed Jun 01, 2005 11:34 am
Posts: 112
Post Re: Circular includes
In case it's useful for anyone else, I ended up making a file "rrequire.i" as so:

Code:
if(is_void(orequire)) orequire = require;

func rrequire(hist, source) {
/* DOCUMENT rrequire, source
  SOURCE should be a scalar string, to be interpreted as a filename like
  "yorick_source.i". This file will be sourced if it hasn't already been.

  Unlike the builtin require, rrequire is recursive-aware. It detects
  dependency loops and will refrain from re-requiring a file if it's currently
  already being required.

  Typically, the builtin require will be replaced by rrequire and the original
  require will be renamed to orequire.
*/
  // If you want to see a dependency tree, manually change this to if(1)
  if(0) {
    write, format="%s", " ";
    for(i = 1; i <= numberof(hist.prev); i++)
      write, format="%s", "| ";
    write, format="%s", source;
    if(anyof(hist.prev == source))
      write, format="%s", " (loop)";
    write, format="%s", "\n";
  }
  if(anyof(hist.prev == source)) {
    // If you want to debug recursive requires, manually change this to if(1)
    if(0) {
      write, "recursive require detected:";
      w = where(hist.prev == source)(1);
      loop = hist.prev(w:);
      write, format="%s", "  ";
      write, format=" %s ->", loop;
      write, format=" %s\n", source;
    }
  } else {
    save, hist, prev=grow(hist.prev, source);
    orequire, source;
    if(numberof(hist.prev) > 1)
      save, hist, prev=hist.prev(:-1);
    else
      save, hist, prev=[];
  }
}

rrequire = closure(rrequire, save(prev=[]));

require = rrequire;


This transparently changes the functionality of require without any change of syntax. As needed, I can temporarily toggle the either of the forced if(0) blocks on to debug and that will allow me to eventually prune out circular dependencies. Once all the loops are gone, I can then stop using rrequire and go back to using the builtin again.

Here's a simpler version without debug code in case it's more helpful:

Code:
if(is_void(orequire)) orequire = require;

func rrequire(hist, source) {
  if(noneof(hist.prev == source)) {
    save, hist, prev=grow(hist.prev, source);
    orequire, source;
    if(numberof(hist.prev) > 1)
      save, hist, prev=hist.prev(:-1);
    else
      save, hist, prev=[];
  }
}

rrequire = closure(rrequire, save(prev=[]));

require = rrequire;


Wed Dec 14, 2011 11:20 am
Profile
Display posts from previous:  Sort by  
Reply to topic   [ 4 posts ] 

Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by STSoftware for PTF.