Thanks for the feedback.
You are correct in your interpretation of my description. Some packages only
publish a blob, and while this should eventually also be handled gracefully, a
first working version of an importer would surely be easier to realize if we
focus on the more well-behaved packages at first.
Using NODE_PATH could work, but in my mind it would just make managing all these
paths more complex, compared to just symlinking the correct packages. Symlinking
might lead to some problems with '..' not evaluating to the right place. Some
research for an approach that works for us might still be required, but it seems
that while the location of where npm places the packages has changed since npm
3, how node loads modules at run-time seems to be more or less the same. Any
scheme we come up with that allows node to run the code would suffice, indeed,
as we wouldn't use or depend on npm's package management functionalities. The
latest folder structure as generated by npm seems as good as any starting point
though.
Seeing as testing would be an important part to ensure that our builds are
functioning correctly, I would propose that a first goal would be the mocha
package. As far as I can see, it has only 15 unique dependencies, none of which
seem too big to grok.
After having a functioning test runner, we could choose a package that makes use
of the test runner. Which packages would be most suitable, I do not know right
now, but I am open to any suggestions/wish-lists.
Another issue that you highlighted is of course the fact that, to us, npm is
more of a binary distribution platform, than an actual repository of code
artifacts. I am well aware of this fact, but in theory there is no reason a
checkout via git should not suffice. Whether it will be possible to properly
pinpoint the checkout for a specific version of each project, and if these
sources are in a functional state, could worst-case scenario be solved with some
elbow-grease when packaging them up.
If recursive imports are not deemed a must-have, maybe we can bump this down to
the stretch goals? It would make sense to chart new territory after having some
experience with Guix under my belt as well ;-)
A potential problem with Gulp, and I am assuming the situation with Grunt is not
much better, is that it (indirectly) depends on an enormous list of
dependencies. I have been keeping half an eye on this mailing-list, especially
with regards to what to do with some of the 'bootstrap-binaries' for Haskell and
other languages. If possible, I would like to not add more fuel to this fire,
but packaging up all the dependencies of the de facto build tools might prove to
be too monumental a task for now. In which case, a temporary solution using a
bootstrapped Gulp/Grunt/both might work. If possible, some advice from people
involved with similar situations would be nice.
The node-build-system seems doable, depending on what we need to cover. Node-gyp
might take a bit of time to properly support, but I will look into this
tomorrow. Most other build steps for npm modules include running some external
program, either binary or a globally installed node package, and shuffling some
files around. When we get the popular build systems to work, the hardest part
(w.r.t. the build system) will have been taken care of.
I will send a revised planning tomorrow.