[Texmacs-dev] Subdirectory per document, metadata, specific svn or git version of local packages maintained per document for archival purposes and development

[Karl Hegbloom]

I bet some of you would enjoy helping me think this all out and see what can be created within and surrounding TeXmacs to facilitate workflows involving creation and management of a series of documents with common macros, services, and look-and-feel, that must remain stable and continue to function even as the stylesheet and document production automation services evolve over the course of the series, necessitating reference to version-sensitive information from frozen and released documents.

I'm thinking of using this for legal writing; obviously best practices indicates writing it in a form that is abstractable to any series of writings with similar but developing styles... e.g. mathematics homework answer workbooks, handouts, quizzes, or exams, civil litigation document management. Once a document is filed for court, it may not be modified. In order to freeze it in place, but make it still load up into TeXmacs and be capable of producing the identical output as it did 2 years ago, the document needs to reference a stable version of the local package.ts files it utilizes, or any scheme programs it needs. For example, I use a legal-filing.ts and casenicks.ts to provide features, services, and a common look-and-feel to all of the documents managed while I learn paralegal writing skills and civil litigation procedure. I think I want each large document that uses local package.ts style files to have its own subdirectory, The subdirectory can contain a file of metadata concerning the document; a bibTeX file would do nicely, right? One for the metadata of this document, and one for the documents actual bibliographic information. (Of course it needs Bluebook or McGill legal citation styles for tables of authorities... or a link with ML-Zotero running in Firefox?)

When I cite statutes or caselaw, I want the PDF to carry a hyperlink to a URL for quick and easy access to that information. The Utah Code is available at a well known URLbase, and is displayed by a program that can use the query-string to find a specified version of the statute, as it is on the day the document is finalized or as it was upon the date pertinent to the discussion regarding a particular statute or rule. Appellate and Supreme Court caselaw does not change once it has been issued. Each have a deterministic URI string that can templated based on variables that can be gotten from a macro's arguments, so \TCS can create a presentable reference to a title, chapter, and section of the statutes, wrapped with an \hlink, and entered into the pertinent table of authorities in case one is to be printed with the document. So far, first iteration, it looks good, but hlinks must be done by hand and I've not figured out how to make my own sort of bibliography or glossary lists for sets of authorities, such as constitutions, federal statutes, state statutes, federal caselaw, state caselaw, journal articles, or textbooks.

Each `part', `chapter', or `...' can be contained within a separate file, all linked with the master document. This breaks the document tree down... I'm hoping it will speed up the editor also. What are other users experiences with that? I think maybe it takes TeXmacs way too long to traverse the document tree. If it does that a lot, and it's a slow or crucially central often-run algorithm, I bet it does that a lot more than it needs to; perhaps some sort of memoization; intelligent caching for shortcuts through the tree?;  But for my own or some generally useful common file organization for complex and compound documents of this nature... I want a subdirectory per document with (?) arbitrary or programatically managed (?) organization above that; Nice to have a tag-cloud and good curation tools, prompts for initial meta-data, automation. Within a document's subdirectory, I want a subdirectory for each exhibit.

The metadata needs to securely record, via use of git features and mandatory commits of document history entries that mark significant calendaring, scheduling, and timelining events, such as document initiation date, document_completion_date, filing_deadline, filing_date, answer_deadline, reply_deadline, submit_for_decision_date, document_type_tag_set (memorandum, motion, answer, reply, interrogatory, request_for_admission, affidavit, of_petitioner, of_respondent), document_name, civil_case_number, court_venue_designation, ...

Perhaps they can be managed in a document management system that uses a revision control system like git for locally produced documents stored in a deterministic filesystem structure, potentially utilizing cryptographicly secure signatures and even some sort of double-custody sign-off feature, and metadata concerning efiling; for document handling, managed review, release, efiling, and archiving protocols; There can be no direct edits of past revisions due to SHA hash etc that git provides; This is something that log-structured record keeping systems which require an audit trail such as court case-histories require, to prohibit edits of previous entries, each is signed and each revision relies on the previous; like a laboratory notebook.

A simple Makefile can do whatever it needs to do to export the latest or a specific set of the version-sensitive inclusions from your git repository; It can easily use templates (autoconf is overkill) to instantiate subdirectories for the {$document_name}_Exhibit_{$symbolic_exhibit_name}.d and each subdirectory will need to be made by hand when an {$document_name}_Exhibit_An_Exhibit_by_This_Name.d/exhibit.tm for exhibit cover-page and summary information, with the running page number, and any files it includes somehow in the PDF output...

I want to insert actual PDF pages from scans or other documents, and have the page numbering increment properly, and superimposed nicely onto each included page in an unobtrusive footer; the footer should probably have a solid white background and black text so the entire footer or relevant portions of it is clearly visible and on top of the stacking order when the page is rendered; It should not require much more work in every day use to add an exhibit or chapter to a document than it takes to invoke a macro that names and pulls in the exhibit cover page and included page-list. Using cut-and-paste, it should be easy to rearrange the finaly order of those chapters and exhibits, and to easily respecify page-inclusion-lists for those exhibits.

Q: What if the included PDF has it's own bookmarks? They should be reparented to be sub-bookmarks of the Exhibit, and landing-point-editted to reflect their correct page-number within the including document... I wonder what it will take to do that? I bet pdftk or something like that can do it with some scripting. Maybe pull the bookmarks and landing zones for the pages of the pdf(s) being included, and associate the bookmark name to the landing zone on the page it references, and use those as input to the routine that rebuilds the bookmarks of the final PDF after it has been written out by TeXmacs? A variables I might like for configuration, if it ever comes up, is: included_pdfs_bookmark_depth_limit.

Sometimes I mark up pages of the PDF with Xournal or an annotation capable PDF reader app, but maybe TeXmacs own drawing functions can do similar things, or more nicely display PDF annotation notes balloon content? It might have the advantage of using the same fonts and scaling as the rest of the document when you write captions on marked up documentary evidence---selected pages from or entire copies of financial declarations, written statements, photographs (possibly requiring special paper for high-fidelity reproduction as an evidencial exhibit before a court, etc.).--- Can a scheme program run from inside TeXmacs when the document is updated manually read a short configuration file that specifies the page-list for an exhibit, count how many pages, and create headings and bookmarks at the correct page number or location within the final PDF. (? PDF/A ?)

So, any ideas? What already exists within TeXmacs that can be utilized as a basis for or part of a document creation and workflow management solution like the above? What already exists as part of the revision control support? The gpg signature support? What are "remotes"? What else?

Karl