Wondeful, Douglas - trying it on some Sugar and XO guides now. :) I'll be sure to not blow a fuse with too much PDF rendering, though. <br><br>Thanks for the hard work on this project!<br>Anne<br><blockquote style="margin: 1.5em 0pt;">
<table><tbody><tr><td>
<img src="http://jwc.midasnetworks.com/blog/wp-content/uploads/2007/12/anne_003_smal-150x150.jpg" width="50" height="50">
</td><td></td><td>
<div style="padding: 5px; font-size: 11px; color: rgb(102, 102, 102); font-family: sans-serif;"><strong>Anne Gentle</strong>
<br>
<a href="mailto:annegentle@justwriteclick.com" target="_blank">annegentle@justwriteclick.com</a>
<div style="font-size: 10px;">
<a href="http://justwriteclick.com" target="_blank">my blog</a> |
<a href="http://xmlpress.net/publications/conversation-community/" target="_blank">my book</a> |
<a href="http://twitter.com/annegentle" target="_blank">Twitter</a>
</div></div></td></tr><tr></tr></tbody></table></blockquote><br><br><div class="gmail_quote">On Thu, Jul 9, 2009 at 8:38 AM, Douglas Bagnall <span dir="ltr"><<a href="mailto:douglas@paradise.net.nz">douglas@paradise.net.nz</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">hello,<br>
<br>
This is just to announce the impending completion of a new PDF maker for<br>
FLOSS Manuals. You can try it at <a href="http://vps504.greenhost.nl/" target="_blank">http://vps504.greenhost.nl/</a> (not all<br>
at once though, or the Netherlands will suffer a power blackout). In<br>
due course it will move to a more memorable url and have a nicer interface.<br>
<br>
I started this because the existing objavi doesn't cope with<br>
bidirectional text, but it is also a step toward the new booki and opens<br>
up possibilities for new sizes and layouts.<br>
<br>
The project's README is attached for people who want a whole lot of<br>
trivial details.<br>
<br>
If you're interested in helping, please subscribe to the tech list at<br>
<a href="http://lists.flossmanuals.net/listinfo.cgi/tech-flossmanuals.net" target="_blank">http://lists.flossmanuals.net/listinfo.cgi/tech-flossmanuals.net</a> -- even<br>
just reporting problems is useful.<br>
<br>
thanks,<br>
<font color="#888888"><br>
Douglas<br>
<br>
</font><br>Objavi2: another FLOSS Manuals publishing system<br>
================================================<br>
<br>
Introduction<br>
============<br>
<br>
FLOSS Manuals books are written and stored as HTML, but are converted<br>
to PDF for printing. Programs which perform this task are called<br>
objavi (pronounced "ob-YAH-vee", as if the J was a Y), after the<br>
Croatian word "objavi!" meaning "publish!".<br>
<br>
the previous objavi, Objavi beta, works very well but is unable to<br>
process bidirectional text and is closely tied to the TWiki software<br>
that FLOSS Manuals intends to abandon. Objavi2 was written to provide<br>
a fully internationalised objavi that is decoupled from other parts of<br>
the FLOSS Manuals system. It was not intended to outdo Objavi beta at<br>
the things that Objavi beta does well, though in some regards it has<br>
and that is OK too.<br>
<br>
Objavi2 is free software, distributed under the version 2 or greater<br>
of the Gnu General Public License. The source can be viewed at<br>
<br>
<a href="http://repo.or.cz/w/objavi2.git" target="_blank">http://repo.or.cz/w/objavi2.git</a><br>
<br>
which also contains instructions for cloning the git repository. If<br>
you want a source tarball without worrying about git, try this link:<br>
<br>
<a href="http://repo.or.cz/w/objavi2.git?a=snapshot;h=HEAD;sf=tgz" target="_blank">http://repo.or.cz/w/objavi2.git?a=snapshot;h=HEAD;sf=tgz</a><br>
<br>
It is primarily written in Python, with a substantial amount of<br>
QSAScript (an ECMAscript variant) and some Javascript, HTML, and CSS.<br>
<br>
Which Objavi should I use?<br>
==========================<br>
<br>
Short answer<br>
~~~~~~~~~~~~<br>
Try both and see which you like, unless the book in question has<br>
right-to-left text, in which case you want Objavi2.<br>
<br>
Details<br>
~~~~~~~<br>
Objavi beta (written in 2008 by Aleksandar Erkalovic) is a TWiki<br>
extension that uses Pisa to make PDFs. Pisa lets you use CSS rules to<br>
avoid widowed or orphaned text and to adjust margins. In other<br>
regards, its CSS support is variable. This means Objavi beta makes<br>
well laid-out books, but people writing style rules need to be aware<br>
of its quirks and use peculiar workarounds to achieve certain effects.<br>
It only works with left-to-right scripts and possibly mis-renders some<br>
of those (due to not understanding combining characters).<br>
<br>
Objavi2 uses Webkit to make PDFs. Webkit is a common web browser<br>
engine, so it interprets CSS in a fairly predictable fashion but also<br>
has almost no concept of paged media. It does not recognise CSS rules<br>
for setting page sizes or margins and has limited support for<br>
controlling page breaks. (There are actually ways in which margins<br>
can be customised with Webkit but Objavi2 does not yet expose them).<br>
Webkit has very well tested Unicode support and it handles<br>
bidirectional text.<br>
<br>
The page-break CSS properties supported by Webkit are<br>
page-break-before and page-break-after, which is sufficient to have<br>
each chapter start on a new page, but not to avoid breaking up<br>
paragraphs in unfortunate ways.<br>
<br>
Objavi2 is somewhat faster than Objavi beta.<br>
<br>
Compatibility<br>
~~~~~~~~~~~~~<br>
The two objavis share no code but have a similar CGI interface, so<br>
sending the same request might result in a PDF being produced<br>
whichever Objavi was installed. This behaviour is inherited rather<br>
than guaranteed, and might fade away.<br>
<br>
<br>
The FLOSS Manuals Book Format<br>
=============================<br>
<br>
FLOSS Manuals source HTML<br>
~~~~~~~~~~~~~~~~~~~~~~~~~<br>
The subset of HTML used in FLOSS Manuals books has been pragmatically<br>
determined rather than specified. The constraints that have shaped it<br>
are that the source must be:<br>
<br>
1. easily producible and editable using the Xinha editor and by hand,<br>
2. printable using Objavi beta,<br>
3. organised into chapters, and<br>
4. conformant to the instincts and habits of the authors.<br>
<br>
This has led to simplified HTML that has the following properties:<br>
<br>
* Each chapter starts with an <h1> heading and contains no other <h1><br>
elements.<br>
<br>
* Each chapter is in a separate file.<br>
<br>
* Fixed width elements such as images are generally no bigger than<br>
600 pixels wide.<br>
<br>
* Inline style, class and id attributes are avoided.<br>
<br>
* Many uncommon or irrelevant tags are avoided.<br>
<br>
* <pre> blocks use less than about 80 columns, though this is<br>
commonly broken.<br>
<br>
* Spurious entities and the like are despised but are left<br>
unmolested in practice unless they cause obvious problems.<br>
<br>
* All of these guidelines are regularly broken if the printed page<br>
looks OK.<br>
<br>
TOC.txt file<br>
~~~~~~~~~~~~<br>
In addition to the HTML chapters, the source of a FLOSS Manuals book<br>
contains a file named TOC.txt which orders the chapters and groups<br>
them into sections.<br>
<br>
The TOC.txt format is quite simple but fiddly to describe and thus<br>
undocumented. An example can be seen here:<br>
<br>
<a href="http://en.flossmanuals.net/pub/Audacity/_index/TOC.txt" target="_blank">http://en.flossmanuals.net/pub/Audacity/_index/TOC.txt</a><br>
<br>
and decoding methods can be found in the Objavi2 source. Much of the<br>
information encoded in the TOC.txt file is useless to Objavi.<br>
<br>
<br>
The objavi process<br>
==================<br>
<br>
Objavi2 starts with the chapters of a book concatenated in order, as<br>
provided by links like this:<br>
<br>
<a href="http://en.flossmanuals.net/bin/view/Audacity/_all?skin=text" target="_blank">http://en.flossmanuals.net/bin/view/Audacity/_all?skin=text</a><br>
<br>
and separately fetches the TOC.txt file described above. Using lxml<br>
(an xml/html library), it finds and numbers chapter headings,<br>
canonicalises image links, and inserts section headings which group<br>
related chapters together. This modified HTML is sent to wkhtmltopdf,<br>
a command-line interface to Webkit that renders a PDF.<br>
<br>
At this point the PDF has no page numbers, no gutters, no table of<br>
contents, and is using a too big paper size. In order to write a<br>
table of contents, the text is re-extracted from the PDF and searched<br>
for invisible tags that were added along with the chapter numbers. (It<br>
is not possible to know what page a chapter will end up on before the<br>
PDF has been laid out). The table of contents thus generated is<br>
combined with other preliminary pages and another PDF is created.<br>
<br>
Pdfedit is used to crop the pages down to size and to shift them<br>
alternately left and right, creating a gutter for the spine of the<br>
book. Then pdfedit is used again to add page numbers to both PDFs,<br>
with lowercase roman numbers being used for the preliminary pages.<br>
<br>
Finally the two PDFs are combined using pdftk and, optionally, spun<br>
180 degrees so they appear upside down. If a right-to-left book is<br>
printed like this on a left-to-right printer, the binding will be on<br>
the correct side.<br>
<br>
Pdfedit and wkhtmltopdf both require an X server to run, for which<br>
Xvfb is used.<br>
<br>
How this differs from Objavi beta<br>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>
Because it is integrated into TWiki, Objavi beta doesn't need to fetch<br>
the book or the TOC.txt over the network, and can instead construct<br>
the html using the TWiki library. Pisa adds page numbers and<br>
generates a table of contents as it goes, and the gutter is set using<br>
its advanced CSS page support.<br>
<br>
<br>
Future plans<br>
============<br>
<br>
There is a TODO in the git repository, but one or two items are worth<br>
expanding.<br>
<br>
It should be easy to add Gecko as an optional layout engine, so people<br>
can choose between the Webkit and Gecko versions. Some languages<br>
might suit one more than the other, and when one grows new paged media<br>
CSS features, Objavi2 will not be stuck with the wrong choice.<br>
<br>
On the other hand, Objavi2 could be tempted into a tighter snuggle<br>
with Webkit, as its front-end wkhtmltopdf is able to generate PDF<br>
outlines and tables of contents as it makes the PDF. The way it does<br>
these things is currently unusable by Objavi2, but it could be<br>
changed.<br>
<br>
Another intriguing but probably stupid possibility would be to embed<br>
Webkit directly in Objavi2 using pyQT.<br>
<br>
<br>
Installation<br>
============<br>
<br>
See the INSTALL file. Apologies for its inadequacy.<br>
<br>_______________________________________________<br>
Discuss mailing list<br>
<a href="mailto:Discuss@lists.flossmanuals.net">Discuss@lists.flossmanuals.net</a><br>
<a href="http://lists.flossmanuals.net/listinfo.cgi/discuss-flossmanuals.net" target="_blank">http://lists.flossmanuals.net/listinfo.cgi/discuss-flossmanuals.net</a><br>
<br></blockquote></div><br>