import string
s1 =  "\xa0x\xa0"
s2 = u"\xa0x\xa0"
print repr(s1.strip())
print repr(s2.strip())
print repr(s2.strip(string.whitespace))

And what we see in the output?

'\ xa0x\ xa0'
u'x'
u'\ xa0x\ xa0'

The second line is different from the others two, and a programmer 99.99% expects that all the three lines should be the same.

Formally, Python is correct. The unicode class of whitespace includes also the unbreakable space. But let’s think further. We can split the whitespace class on two further subclasses, (in TeX terms) droppable glue and undroppable kern. And split() shouldn’t touch the latter.

Fortunately, there is a workaround by specifying string.whitespace explicitly.

Discussion: pointing kpathsea to the correct texmf tree.

Summary: texmf.cnf defines search paths in terms of SELFAUTO* variables (at least, in vanilla TeXLive and MacTeX 2009). Therefore, kpathsea looks for files relative to the location of the calling program.

Therefore, to find the files, the calling program should be located in the right place or all the AUTO-based search paths should be redefined. For me, the following wrapper implements the second idea:

—– textool.sh: begin

#!/bin/bash

for i in         \
  TEXMFMAIN      \
  TEXMFDIST      \
  TEXMFLOCAL     \
  TEXMFSYSVAR    \
  TEXMFSYSCONFIG \
  TEXMFCNF       
do
  export $i=`kpsewhich -expand-var "$"$i` 
done

"$@"

Running kpsewhich in a wrong location doesn’t work:

$ ./kpsewhich psfonts.map
warning: kpathsea: configuration file texmf.cnf not found in
these directories: ...
$ export export TEXMFCNF='{/usr/local/texlive/2009/.....
$ ./kpsewhich psfonts.map
$

But the wrapper helps:

$ textool.sh ./kpsewhich psfonts.map
/usr/local/texlive/2009/texmf-var/fonts/map/dvips/updmap/psfonts.map
$ textool.sh ./a.out 
psfonts.map=/usr/local/texlive/2009/....../psfonts.map
$

Existing tools consider XML to paper conversion as one black box step and provide control only through predefined options. With this approach, tuning the layout of output documents is a burdensome task.

The paper advocates a new workflow for XML to paper conversion, in which a separate step allows the user to fine control the layout. The changes made by the user are remembered and later can be automatically re-applied during publishing an updated version of the document.

A possible technical implementation for the workflow is suggested. TeX is used as a typesetting engine. XML to TeX conversion is made using XSLT and TeXML. The management of changes is performed by diff and patch tools.

The LaTeX code to compile is:

\documentclass{minimal}
...SOMETHING...
\begin{document}
$$\sin(\alpha+\beta) = \sin α \cos β + \cos\alpha \sin\beta$$
$$\int\!\!\!\int_D dx\,dy$$
\end{document}

Note that “alpha” and “beta” are written both as command sequences and as unicode characters.

The part marked “…SOMETHING…” is a placeholder for setup.

Fast, no so wrong way

If the goal is only to:

* allow the greek letters as unicode symbols, and
* it is not important which math font is used,

then the simplest solution is:

* make the greek letters active,
* define the greek letters as the corresponding commands:

Replace the placeholder “…SOMETHING…” with:

\catcode`\^^^^03b1=13
\let^^^^03b1=\alpha
\catcode`\^^^^03b2=13
\let^^^^03b2=\beta

Repeat it for the all possible letters.

unicode-math package

This package is not yet released on CTAN. Latest developmental and archived historical versions of this package are available from GitHub: http://github.com/wspr/unicode-math.

Given that the author, Will Robertson, is also the author of such packages as fontspec, xltxtra, euenc etc, I think that unicode-math will be a standard way to have unicode math in XeLaTeX.

After installing this package, I succesfully compiled the test document using the following “…SOMETHING…”:

\usepackage{unicode-math}
\setmathfont{[Asana-Math]}

The font Asana-Math can be found on TeX Live 2009 DVD (and probably on 2007 and 2008 too). The square brackets around the name forces xelatex to search for a .otf/.ttf-file instead of getting the font from OS. In my case, xelatex somehow found the font automatically, I didn’t need to give the full path.

A quick search gave a migration notes of Audacity: CVS To SVN Migration. To get a better understanding what is happening, I read the following documents:

* Version Control with Subversion / Converting a Repository from CVS to Subversion
* cvs2svn Documentation

The dump was easy to produce:

$ rsync -av --delete-delay rsync://PROJECT.cvs.sourcefor
   ge.net/cvsroot/PROJECT/* PROJECT-cvsbackup/

$ cvs2svn --dumpfile=svndump PROJECT-cvsbackup > cvs2svn.log

The task of importing the dump was more challenging, but also easy solved thanks to the sourceforge documentation: Managing your Subversion Repository with adminrepo

This moves the dump on the sourceforge servers and logs in:

$ rsync -aiv svndump USER,PROJECT@web.sourceforge.net:.
$ ssh -t USER,PROJECT@shell.sourceforge.net create

And this what happens after logging in:

$ mv /home/groups/P/PR/PROJECT/svndump .
$ adminrepo --checkout svn
$ rm -rf /svnroot/PROJECT/*
$ svnadmin create /svnroot/PROJECT/
$ svnadmin load /svnroot/PROJECT < svndump
$ adminrepo --save

Project options

Remembered after writing this post: one must to activate subversion before doing the conversion:

Project admin -> Feature settings -> Available Features

After conversion, disable CVS.

Private Portfiles

For one-computer use, there is no need for a complete setup. There is enough to say something like “sudo port install -d -v” in the directory with a Portfile.

For details, see the official documentation. For introduction, read the book “Mac OS X Tiger for Unix Geeks”, the chapter “Creating and Installing Packages”, section “Creating DarwinPorts packages”.

What is required

1) I use the following structure for the local repository:

files/
  ... distribution archives ...
ports/
  PortIndex
  myorg/
    port1
    ...
    portN
  _resources/
    port1.0/
      fetch/
        mirror_sites.tcl
      group/
        syspython25-1.0.tcl
        syspython26-1.0.tcl

2) Access to the local ports server is available only through rsync (01.02.2010: probably things changed). A possible content of /etc/rsyncd.conf:

[orgmacports]
  path = ....../ports
  comment = My Organization MacPorts, the port files
  exclude = .** **~

To check: “rsync rsync://myhost.mydomain/orgmacports/“. This lists the content of the directory “ports“.

3) Unfortunately, macports system can’t download files through rsync (01.02.2010: maybe already able to). Therefore, share the content of “files” through FTP or HTTP.

4) Define the repository in “/opt/local/etc/macports/sources.conf“:

rsync://rsync.macports.org/release/ports/ [default,nosync]
rsync://myhost.mydomain/orgmacports/

Note that I’ve added “nosync” to the main macports reposotory: playing with the local repository, one says “sync” often, and no-sync for the main repo safes time.

To check, execute “$ sudo port -v sync“. The output should contain a note about synchronitzation with the local repository.

Inside _resources

I do not remember exactly why I had to add the file “mirror_sites.tcl” (nothing worked without this file). Also, I removed a lot of code, so the current version looks so:

# $Id$
namespace eval portfetch::mirror_sites { }

set portfetch::mirror_sites::sites(macports) {
    http://svn.macports.org/repository/macports/distfiles/
    http://svn.macports.org/repository/macports/distfiles/general/:nosubdir
    http://svn.macports.org/repository/macports/downloads/
}

set portfetch::mirror_sites::sites(macports_distfiles) {
    http://distfiles.macports.org/:mirror
    http://trd.no.distfiles.macports.org/:mirror
    http://arn.se.distfiles.macports.org/:mirror
}

The files “syspython25-1.0.tcl” and “syspython26-1.0.tcl” are my private groups, which define the settings to compile modules for the system python.

Abstract

TeXML: XML to PDF through LaTeX

TeXML is an approach to XML to PDF conversion, an alternative to the XSLT/FO way. The characteristic features are:

* LaTeX is used as a typesetting engine.
* XML logical markup is converted to LaTeX logical markup.
* The low-level formatting details are hidden in LaTeX style files, not in conversion scripts.
* XML to LaTeX conversion scripts are straightforward thanks to an intermediate XML-based language.
* Generated LaTeX code is human-friendly and ready for modification.

To be a real alternative to the established standard, TeXML needs unique competitive advantages. The main are:

* Poor quality auto-layout artifacts in generated PDFs can be fixed by editing LaTeX code.
* Separation of work is encouraged: programmers do programming, and typesetting specialists do layout definition.
* The process is based on open-source tools.

The paper is written from the XML perspective. Only basic knowledge of LaTeX is assumed. TeXML is compared with alternative TeX-based XML-to-PDF projects. Practical experience of using TeXML in production is given. Problems and listed together with the solutions. An appendix provides an example of a simple publishing project, with step-by-step instructions how to convert XML to PDF using TeXML approach.

Reviews

Neutral

I think that TeXML is not new project and given it’s non-mainstream technology I don’t think that XML Prague is right venue for having presentation about it.

Reject

There’s nothing new in this paper that hasn’t been said in the last 10 years. Furthermore there is no paper attached.

Strong Reject

lacks enough detail to make a positive decision

Reject

Not matching this year’s topics.

Reject

The paper presents a tool for generating PDF output from XML via an XML representation of LaTeX called TeXML, which seems to be a useful light-weight approach for producing high-quality output in the kinds of applications where TeX excels. This topic, however, is a bit distant from the conference focus for this year.

Conclusion

My habit of sending an abstract only and writing the paper later has failed here: 2 rejects of 5 reviews. As of not matching this year’s topics, usually it doesn’t matter.

For me, it seems the failure is due to a bit of unluck. Being of non-mainstream (thanks to the first reviewer for pointing it to me), I get reviewers who a bit distant from publishing XML on paper and don’t see any use for TeXML. Contrary, the real users of TeXML are excited with it, and I believe the users more than reviewers.

TODO:

Re-write the abstract and paper so that even reviewers not in topic understand that something important happens.

Finally release the working DocBook stylesheets to be less non-mainstream.

Rework the site, add more new documentation.

It seems that skak is the package to typeset chess using LaTeX. (See also an earlier post, “chess publishing in latex, starting”.)

Unfortunately, sometimes skak ignores castling moves. I’m going to produce a minimal example and submit a bug report. But at the moment, in case case of such error, use the command \fenboard to correct a position.

My class diakarte imports skak and defines two new commands:

\therest{…}: the argument is centered horizontally using \centerline and then centered vertically in the rest of the text flow.

\restcomment{…}: the argument is centered horizontally using \centerline and then attached to the previous \therest.

Something like this:

\therest{\showboard}
\restcomment{What now?}

A complete example demonstrates also advanced usage. It is included in the zip-file with diakarte package.

If you’ve found the package useful, please drop me a note in comments.

After different attempts, the best so far option is found:

* I use thick paper index cards of size A6.
* To hold all the card together, I use foldback clips.

Printing, preprinting

Many printers have a special slot to put thick paper into the printer.

I though it is enough to 1) put an A6 card to the slot, 2) print an A6 pdf/ps-file. Unfortunately, no. Regardless of the card size, the printer prints the document like on an A4 paper. As result, the text appears outside the card.

After spending time experimenting with CUPS, I gave up. The next easiest solution is to correctly position an A6 sheet on an A4 sheet.

After some calculations: if I want 1cm margins on A6, I should set the following margins for A4:

* left, right: 64mm
* top: 1cm
* bottom: 159mm

These values are best put to a template in your favourite text processors.

Just for a case, A-sizes: http://en.wikipedia.org/wiki/A4_paper

Part 2

The letter continues:

I’ve diagnosed the problem. The cause is a change in the encoding option of the TexML converter, from -eISO-8859-1 to -eUTF-8, as expected by XeTeX. Previously some unicode characters were automagically converted to the corresponding LaTeX mathematical symbols, but now they are just copied to the generated LaTeX code.

I’ve solved the problem myself by a quick-and-dirty hack of texmlwr.py. A new conditional section in writech(...) before the chained tests for TeXML modes forces the conversion inside math mode, regardless of the particular output encoding.

263c263
 <     # First attempt to write symbol as-is
---
 >     # First attempt to write symbol as-is  **** IF NOT IN MATH MODE ****
264a265,276
 >     if self.mode == MATH:
 >       #
 >       # Math mode, lookup math map
 >       #
 >       try:
 >         chord = ord(ch)
 >         if chord > 255:                            # no Latin-1
 >           self.stream.write(unimap.mathmap[chord])
 >           return                                             # return
 >       except:
 >         pass
 >

My coment

So, XeTeX (and probably other TeXs, I think) doesn’t interpret the unicode math correctly. The solution is to avoid the unicode math and use the corresponding escape sequences instead.

I’m afraid it’s the only available solution right now. However, I vaguely remember discussions in XeTeX mailing list and articles in TUGboat, therefore I’d like to spend some time in looking for an alternative. Probably a magic option for XeTeX cures the problem.

Finally, I asked for a few examples of TeXML, generated TeX and PDF files (both correct and not). Hopefully, I’ll have a bit of time to find a better solution.

26.03.2010, solution

unicode math in xelatex

Unfortunately, I got technical problems. Fortunately, solved them.

The main issue is that my Mac Mini can’t read the CD. Therefore, I tried to rip the disk and use the image. To my big surprise, it worked. No idiotic anti-copy protection.

Ripping a CD. Probably user-friendly tools exist, but I used something like this:

$ sudo umount /Volumes/Fritz\ und\ Fertig
$ sudo dd if=/dev/disk1s1s2 of=ff.iso

To mount the image, just click on “ff.iso” in Finder.

But can you click on the image in Simplified Finder (parental control works)? Probably yes, but I don’t know how. Therefore, I wrote a wrapper script “FritzUndFertiz.command“:

#!/bin/sh

open /Users/.....path.to.../ff.iso
sleep 5
open '/Volumes/Fritz und Fertig/Fritz - OS X/Fritz und Fertig'

To run the script, set the executable flag (”chmod +x FritzUndFertiz.command” in the console). Move the script to “MyApplications”, try to run, allow all the required programs, and –

The child can run Fritz&Fertig without assistance.

So, the step 1. Print some game or some position.

I opened “LaTeX Graphics Companion” (first edition, 1999, probably I should upgrade) and read about chess. The package I needed is named (surprise!) chess.

It’s a real monster. You put the moves in, and the package automatically traces how the figures move (including en passant etc). At any moment, it can typeset a diagramm. Very impressive!

It’s an overkill for me. All I need is diagramms. And after I found that the symbol “|” has a special meaning and can conflict with the usual use, I decided to find something else.

There is a number of chess-packages on CTAN. However, it was easy to minimize the choice. As the important question for me is “how to select/change a chess font”, I immediately was interested in the actively developing packages:

* enpassant, a collection of type1 chessfonts ready for TeX, and
* chessfss, chess fontselection scheme.

The author is Ulrike Fischer, the both packages are not in TeX Live, only on CTAN. In the package notes, it is said that the font collection is supported by the package “skak”:

* skak, fonts and macros for typesetting chess games.

The package “skak” seems an improvement of “chess”. It is included in TeX Live. The package provides sample files as documentation, this is enough to start using the package. The first steps are even easier with this program:

* Pgn2ltx - A program for converting PGN (Portable Game Notation) data into a LaTeX input file. By Dirk Baechle, GPL2.

At the end, one more useful link:

* EN PASSANT - Nørresundby Chess Club. En Passant is a chess site with free chess software and resources for chess publishing like chess fonts, diagram utilities, graphics and more. En Passant is the homepage of Nørresundby Chess Club in Denmark.

TODO: check that I really can change chess fonts.

I.

\RequirePackage{grffile}

Without this package, code

\includegraphics{ANSIboat.v1.pdf}

! LaTeX Error: Unknown graphics extension: .boat.v1.pdf.

II.

\def\includegraphicsII[#1]#2{%
\IfFileExists{#2}{\includegraphics[#1]{#2}}{%
\typeout{! LaTeX warning: File `#2' not found.}%
\includegraphics[#1]{dummy.pdf}}}

If there is no image file, LaTeX

* annonces this problem to the console and the log file,
* uses the image “dummy.pdf” instead

III.

\catcode`\_=11
\edef\ImageUnderscore{_}
\catcode`\_=8
\def\Image[#1]#2{{%
\let\_=\ImageUnderscore
\includegraphicsII[#1]{#2}}}

This allows to specify a file name with the underscore symbol using the standard LaTeX escaping rules:

\Image{ANSI\_boat.pdf}

instead of

\includegraphicsII{ANSI_boat.pdf}

Benefit: no need for special escaping rules in an automatic XML to LaTeX converter.

Actually, the code must be extended, to include “\{”, “\textless” and other special symbols, but it’s TODO.

import foo-bar as foo_bar
foo_bar.callme()

After trials and errors, the desired magic found:

foo_bar = __import__('foo-bar', globals(),  locals(), [], -1)
foo_bar.callme()

See Python Documentation for details: in some cases you don’t want to use __import__.

The best solution is, obviously, to rename file. But I wanted to find an answer to this importing challenge.

In smb.conf:

security = share
guest account = nobody

[my-svn]
  comment = SVN files
  path = /path/to/svn
  read only = yes
  public = yes
  guest ok = yes

call child1.bat
call child2.bat

What happens after ‘child1.bat‘ is finished?

I reasonably expected that ‘child2.bat‘ is executed. In a ideal world, yes.

In the real world, the first script contains the command ‘exit‘, and this commands not only exists from ‘child1.bat‘, but also terminates ‘main.bat‘. Therefore, ‘child2.bat‘ isn’t executed. I don’t have words to comment this behavior.

As a workaround, I constructed the following. Not sure it’s correct, but right now it works for me.

start /B /wait child1.bat
start /B /wait child2.bat

And the order of the arguments is important. “/B wait” makes the trick, “/wait /B” doesn’t.

From documentation:

* /B: Start application without creating a new window. The application has ^C handling ignored. Unless the application enables ^C processing, ^Break is the only way to interrupt the application
* /WAIT: Start application and wait for it to terminate

1. Backup!

2. Somewhere in the vmware installation directory, there is a tool named “vmware-vdiskmanager.exe“. There is a lot of options, for resize use “-x“:

vmware-vdiskmanager.exe -x 8Gb ...path.to.vm..../vm_disk.vmdk

(hint: type the command manually, do not copy/paste from web. Otherwise you can get typographical spaces or minuses instead of the usual ones.)

3. The disk is extended, now it’s time to update the partition table and the disk “C” to match the new size of the disk. Several options are availabe, they are listed in vmware’s knowledge base:

VMware Virtual Disk Manager Does Not Expand Partitions
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1647

I tried GParted LiveCD and it worked:

GParted — Live CD/USB/PXE/HD
http://gparted.sourceforge.net/livecd.php

GParted run automatically, I had no problems to use it.

4. When Windows started, it noticed that something strange had happended with the disk and executed “chkdsk”. The tool validated that everything is in order, and Windows started as usual. With a bigger system disk.

*** glibc detected *** sylpheed: double free or corruption (out): 0x0839e818 ***

The right thing is to trace the application and submit a bug report to the author, but I found a fast workaround. Set an environment variable before running the program:

export MALLOC_CHECK_=0