Unfortunately Shipwright’s documentation is lacking, so I thought I’d keep some notes about my process in hopes that it will help others in creating a Shipwright package (or vessel, or repository, or whatever it’s actually called). These notes were written with version 2.2.1.

By the way, I’m only showing output from Shipwright and not from any of the other commands I run. A command run on the command-line is prefixed with a %.

I first started by creating a directory (oracdr-perl-modules) that would hold everything. Then I changed into that directory and initialized a git repository:

 % cd oracdr-perl-modules
 % git init

I then initialized my Shipwright repository and commited the initial state to my git repository:

 % shipwright create -r fs:`pwd`
 % git add *
 % git commit -m 'Shipwright initialization.'

I am not using the git backend for Shipwright, as I will be handling my git repository manually.

As I will be making tweaks to modules to make sure they build correctly, pass tests, etc., I created a vendor branch that would hold the un-modified modules as downloaded from CPAN:

 % git branch vendor
 % git checkout vendor

Then I started adding modules from CPAN:

 % shipwright -r fs:`pwd` import cpan:Proc::Simple
 % git add *
 % git commit -m 'Added Proc::Simple.'

These three steps were repeated for all of the CPAN modules I needed.

Once I finished importing the CPAN modules, I switched to my master branch and merged from vendor, since I want my vendor branch to reflect the “base” state of modules as obtained from the vendor, and my master branch to hold any local changes I have to make to the modules to get them to build properly:

 % git checkout master
 % git merge vendor

Now it was time to make tweaks to some of the modules. As an example, ExtUtils::F77 doesn’t support the g95 Fortran compiler, so I had to make some changes to the F77.pm file:

 % cd sources/cpan-ExtUtils-F77/vendor
 % xemacs F77.pm

Once my changes were complete, I checked in the modified file:

 % git add F77.pm
 % git commit -m 'ExtUtils::F77: Tweaks to support g95.'

I had to add some compilation options for Astro::FITS::CFITSIO, adding a command-line option when running perl Makefile.PL and tweaking Makefile.PL. To do this, I modified the build file:

 % cd ../../../scripts/cpan-Astro-FITS-CFITSIO
 % xemacs build

As I needed to add a command-line option to the Makefile.PL line, I edited the configure: line to read:

configure: %%PERL%% Makefile.PL OPTIMIZE=-O LIB=%%INSTALL_BASE%%/lib/perl5/ PREFIX=%%INSTALL_BASE%% %%MAKEMAKER_CONFIGURE_EXTRA%%

In this case, I added OPTIMIZE=-O.

I then edited the Makefile.PL:

 % cd ../../sources/cpan-Astro-FITS-CFITSIO/vendor
 % xemacs Makefile.PL

Then, another commit to the git repository:

 % git add Makefile.PL ../../../sources/cpan-Astro-FITS-CFITSIO/build
 % git commit -m 'Astro::FITS::CFITSIO: Add OPTIMIZE=-O to Makefile.PL command-line, edit Makefile.PL to know about $ENV{STARLINK_DIR}'

At this point I thought I’d test out the build process, making sure that the modules I’d imported passed their tests. If they didn’t, I could either fix the tests or patch modules (or, in a worse-case scenario, tell Shipwright to skip testing specific modules). I had to make a clone of my git repository in a different directory:

 % cd ~/development
 % mkdir shipwright-test
 % cd shipwright-test
 % git clone file:///home/bradc/development/oracdr-perl-modules

Testing time! I’m testing with a private build of Perl, which is located in $STARLINK_DIR/Perl/bin/perl, thus the extra preamble. If I was just using the system Perl I’d leave that out.

 % cd oracdr-perl-modules
 % $STARLINK_DIR/Perl/bin/perl bin/shipwright-builder
no default install-base, will set it to /tmp/vessel_oracdr-perl-modules-pHWh30/oracdr-perl-modules
Building cpan-Proc-Simple
Building cpan-Tk

During Tk‘s tests it hung up. I’ve seen this before, so I cancelled the build by hitting Ctrl-C. I changed to my original Shipwright directory, deleted the offending test file, and updated my repository:

 % cd ~/development/oracdr-perl-modules/sources/cpan-Tk/vendor/t
 % git rm font.t
 % git commit -m 'Tk: font.t test locks up, remove it'

Then back to my Shipwright test directory and continue on merrily. I first cleaned up the Shipwright build by adding --clean — this might not be necessary, but it allows things to start from a fresh slate. When building XS modules like Tk sometimes header or object files can get into a state, so cleaning them up before rebuilding is good practice, I’ve found.

 % cd ~/development/shipwright-test/oracdr-perl-modules
 % git pull
 % $STARLINK_DIR/Perl/bin/perl bin/shipwright-builder --clean
 % $STARLINK_DIR/Perl/bin/perl bin/shipwright-builder
Building cpan-Tk
Use of uninitialized value in string at bin/shipwright-builder line 443.
build cpan-Tk test part failed.
 at bin/shipwright-builder line 456
        main::_install('cpan-Tk', 'GLOB(0x12f73e10)') called at bin/shipwright-builder line 346
        main::install() called at bin/shipwright-builder line 241

Another error. Rerunning with --verbose^[1] shows that the problem is with the t/listbox.t test, which I removed in a similar fashion to the t/font.t test. Update the repository, clean up, and start again:

 % git pull
 % $STARLINK_DIR/Perl/bin/perl bin/shipwright-builder --clean
 % $STARLINK_DIR/Perl/bin/perl bin/shipwright-builder
Building cpan-Tk
Building cpan-Params-Validate
Building cpan-Class-Singleton
Building cpan-Compress-Raw-Zlib
Building cpan-Compress-Raw-Bzip2
Building cpan-IO-Compress
Building cpan-IO-Zlib
Building cpan-Package-Constants
Building cpan-Archive-Tar
Building cpan-Regexp-Common
Building cpan-Pod-Escapes
Building cpan-Pod-Simple
Building cpan-podlators
Building cpan-Devel-Symdump
Building cpan-Pod-Coverage
Building cpan-Test-Pod-Coverage
Building cpan-Test-Pod
Building cpan-Test-Portability-Files
Building cpan-Pod-Readme
Building cpan-version
Building cpan-ExtUtils-CBuilder
Building cpan-Digest-SHA
Building cpan-Module-Signature
Building cpan-ExtUtils-ParseXS
Building cpan-Test-Harness
Building cpan-Module-Build
Building cpan-DateTime-TimeZone
Building cpan-List-MoreUtils
Building cpan-DateTime-Locale
Building cpan-DateTime
Building cpan-Time-Piece
Building cpan-IO-Tee
Building cpan-URI
Building cpan-HTML-Tagset
Building cpan-HTML-Parser
Building cpan-libwww-perl
Building cpan-Tk-TextANSIColor
Building cpan-XML-Parser
build cpan-XML-Parser make part failed.
 at bin/shipwright-builder line 456
        main::_install('cpan-XML-Parser', 'GLOB(0x116f0e10)') called at bin/shipwright-builder line 346
        main::install() called at bin/shipwright-builder line 241

Well, that’s a bit more successful. Running with --verbose showed that the expat.h file couldn’t be found. I then determined that XML::Parser is needed by SOAP::Lite, which we don’t really need after all, so I switched to my Shipwright directory, changed to the vendor branch, removed SOAP::Lite, changed back to the master branch, then merged in the changes:

 % cd ~/development/oracdr-perl-modules
 % get checkout vendor
 % shipwright -r fs:`pwd` delete cpan-SOAP-Lite
 % shipwright -r fs:`pwd` delete cpan-XML-Parser
 % git add *
 % git rm -r scripts/cpan-SOAP-Lite
 % git rm -r scripts/cpan-XML-Parser
 % git rm -r sources/cpan-SOAP-Lite
 % git rm -r sources/cpan-XML-Parser
 % git commit -m 'Remove SOAP::Lite and XML::Parser.'
 % git checkout master
 % git merge vendor

Note the change in naming convention when using shipwright delete versus shipwright import! For CPAN modules, you import using cpan:SOAP::Lite and delete using cpan-SOAP-Lite.

Now, back to the test directory to start again:

 % cd ~/development/shipwright-test/oracdr-perl-modules
 % git pull
 % $STARLINK_DIR/Perl/bin/perl bin/shipwright-builder --clean
akule % $STARLINK_DIR/Perl/bin/perl bin/shipwright-builder
Building cpan-Task-Weaken
Building cpan-DateTime-Format-Strptime
Use of uninitialized value in string at bin/shipwright-builder line 443.
build cpan-DateTime-Format-Strptime test part failed.

This error is because of DateTime::Locale. You’ll note that the most recent version (as of this writing) is 0.43. Apparently this version causes one of the tests in DateTime::Format::Strptime to fail, so we need to tell Shipwright that we only want version 0.42 of DateTime::Locale. In our Shipwright repository we do:

 % git checkout vendor
 % shipwright update -r fs:`pwd` cpan-DateTime-Locale --version 0.42
imported with success
updated with success

Don’t let those two lines fool you! Nothing has actually happened here^[2]. If you run the last command with -l info you’ll see a message stating:

path scripts/cpan-DateTime-Locale alreay exists, need to set overwrite arg to overwrite

I wouldn’t call this “success”, so I filed a bug^[3]. Turns out the overwrite argument isn’t implemented (another bug filed) so I just deleted DateTime::Locale and imported version 0.42:

 % shipwright -r fs:`pwd` delete cpan-DateTime-Locale
 % shipwright -r fs:`pwd` import cpan:DateTime::Locale --version 0.42
 % git add *
 % git commit -m 'Revert DateTime::Locale to v0.42 to...'
 % git checkout master
 % git merge vendor

At this point I’m not going to show every little error that popped up. Let’s skip ahead to the end:

Building cpan-Astro-FITS-CFITSIO
install finished, the dists are at /tmp/vessel_oracdr-perl-modules-GwEUP5/oracdr-perl-modules

Success!

Sort of. At this point I’ve only done the CPAN modules. Now it’s time to do some of our own modules. Because our modules aren’t stored in separate git repositories, I have to make a tarball of each one and import that:

 % shipwright import -r fs:`pwd` file:/home/bradc/development/starlink/perlmods/Starlink/Config/Starlink-Config-1.01.tar.gz

Starlink-Config-1.01.tar.gz is the result of doing perl Makefile.PL && make dist in our starlink/perlmods/Starlink/Config directory, which is the standard method of creating a tarball for distribution. Shipwright can import these directly.

Unfortunately when you import a module using a tarball, Shipwright doesn’t seem to realize that it exists for the purposes of being an installation requirement for future modules. For example, our Astro::Telescope module depends on Astro::SLA. I imported Astro::SLA from a tarball, then tried to import Astro::Telescope. Instead of realizing that Astro::SLA was already imported, Shipwright imported it again from CPAN. This is a particular problem when you’ve imported a module that doesn’t exist on CPAN, as Shipwright tries to find it, then fails to, and then crashes with a scary error:

 % shipwright import -r fs:`pwd` file:/home/bradc/development/starlink/perlmods/Starlink/HDSPACK/Starlink-HDSPACK-undef.tar.gz
CPAN related output will be at /tmp/shipwright_cpan.log
Use of uninitialized value in numeric eq (==) at /local/perl-5.8/lib/site_perl/5.8.8/Shipwright/Source/CPAN.pm line 85.
invalid source: can't find 'NDF' in your CPAN mirror(s) [http://cpan.mirror.facebook.com http://cpan.knowledgematters.net http://cpan.yahoo.com]. at /local/perl-5.8/lib/site_perl/5.8.8/Shipwright/Source/CPAN.pm line 97
        Shipwright::Source::CPAN::run('Shipwright::Source::CPAN=HASH(0x789c8d0)') called at /local/perl-5.8/lib/site_perl/5.8.8/Shipwright/Source/Base.pm line 441
        Shipwright::Source::Base::_follow('Shipwright::Source::Compressed=HASH(0x6ca0770)', '/tmp/shipwright_source_gIMm9O/Starlink-HDSPACK-undef') called at /local/perl-5.8/lib/site_perl/5.8.8/Shipwright/Source/Compressed.pm line 33
        Shipwright::Source::Compressed::run('Shipwright::Source::Compressed=HASH(0x6ca0770)', 'copy', 'HASH(0x661ac80)') called at /local/perl-5.8/lib/site_perl/5.8.8/Shipwright/Script/Import.pm line 157
        Shipwright::Script::Import::run('Shipwright::Script::Import=HASH(0x639b500)', 'file:/home/bradc/development/starlink/perlmods/Starlink/HDSPA...') called at /local/perl-5.8/lib/site_perl/5.8.8/App/CLI/Command.pm line 53
        App::CLI::Command::run_command('Shipwright::Script::Import=HASH(0x639b500)', 'file:/home/bradc/development/starlink/perlmods/Starlink/HDSPA...') called at /local/perl-5.8/lib/site_perl/5.8.8/App/CLI.pm line 79
        App::CLI::dispatch('Shipwright::Script') called at /local/perl/bin/shipwright line 16

Did you spot the key piece of information there? Here it is:

invalid source: can't find 'NDF' in your CPAN mirror(s) [http://cpan.mirror.facebook.com http://cpan.knowledgematters.net http://cpan.yahoo.com]. at /local/perl-5.8/lib/site_perl/5.8.8/Shipwright/Source/CPAN.pm line 97

Even though I’ve already imported NDF to this Shipwright repository, Shipwright can’t find it. I’ve filed a bug about this one^[4].

The only work-around I can see at this point is to add --no-follow to my imports. This means that I have to have figured out the module requirements and dependency tree before importing every module from this point, which luckily I’ve already done. However, if this had been early in the process and I was relying on Shipwright to figure all of this out for me (which it’s supposed to do) I’d be screwed.

I found that the NDF module was missing a file in its MANIFEST so it wouldn’t build properly. I fixed the module, recreated the tarball, and deleted and imported the module again:

 % shipwright delete -r fs:`pwd` NDF
 % shipwright import -r fs:`pwd` file:/home/bradc/development/starlink/perlmods/Starlink/NDF/NDF-1.48.tar.gz

Now unfortunately when you do this sort of thing Shipwright thinks you want to build this module at the end, when actually you just want to build it where it was before. If you’ve already imported a bunch and want to go back and fix a module in this way, things will build out of order and your dependency tree will be all screwed up. Since I’m keeping this repository in git, I just have to revert a couple of files:

 % git checkout -- shipwright/order.yml
 % git checkout -- scripts/NDF/require.yml

The shipwright/order.yml file tells Shipwright which order to build modules in, and since we want to keep that order, we just revert it to lose the changes imposed by the delete/import pair of commands. The scripts/NDF/require.yml file lists requirements for NDF — I don’t know if this is used by Shipwright anywhere, but we might as well revert it anyhow.

I’m skipping ahead here to the point where I’ve ironed out all the kinks in my Shipwright repository, and it’s time to create a vessel on each platform and install the modules in my Starlink distribution directory. For future reference, Starlink has Perl installed in /star/Perl, so that’s the base directory for this case.

I start by checking out the repository via git:

 % git clone file:///net/akule/export/data/bradc/backup/development/oracdr-perl-modules perlmods-nanahope

Once this is done, I create my vessel:

 % cd perlmods-nanahope
 % /star/Perl/bin/perl bin/shipwright-builder --install-base /tmp/shipwright-vessel

This goes about and, since I’ve fixed up all of the tests and compiles and whatnot, creates the vessel in /tmp/shipwright-vessel.

Because the library directory structure doesn’t match that for Perl itself, I can’t install directly into /star/Perl like I’d want. Instead I have to change to my /star/Perl/lib/perl5/site_perl/5.10.0 directory, then copy the Shipwright-created files over:

 % cd /star/Perl/lib/perl5/site_perl/5.10.0
 % rsync -avz /tmp/shipwright-vessel/lib/perl5/ ./

And I’m done! Well, not quite, as I have some scripts in /tmp/shipwright-vessel/bin that need copying over. However, those scripts are merely wrappers around calls to a Shipwright-specific script. If you’re just installing into a Perl tree, you should copy the files in /tmp/shipwright-vessel/bin-wrapped into your Perl bin directory instead, as it keeps things clean.

1. sunnavy, the author of Shipwright, notes below that a build.log file is written, and that contains the error messages. Just check that for the error messages instead of having to re-run with --verbose!
2. Actually, this whole section doesn’t apply to version 2.3 of Shipwright, as this version bug has since been fixed.
3. Like I said, this bug was fixed in Shipwright v2.3.
4. …which has since been closed because the developer doesn’t consider this a bug. You can see his reply and rationale by clicking that link.

First question was a two-parter: what do you want to work on next year and what don’t you want to work on next year?

First answer is for the second question: PHP. Second answer for the first question: he wants to be (or make) the Red Hat of CPAN. He wants to take the 5000 modules and 2000 developers and have one front-end where things just work. When you’re using a module from CPAN you don’t know what you’re going to get, as modules change all the time with new releases. Schwern wants to change this somehow.

Regarding PHP, Schwern is working with PHP full-time. He says that the good thing about PHP is that when you download it, it just works right out of the box. You don’t have to worry about modules and plugins and whatnot. This leads to PHP’s bad side: global namespace pollution. There are 75 built-in functions to deal with arrays and 13 different sort functions. Crazy!

He wants to declare ExtUtils::MakeMaker finished. Go use Module::Build instead!

A while back Damian Conway said something about Perl 5 + i, which was kind of a joke in that Damian way, but Schwern went and created perl5i. It modifies perl5 “so it fits how it works in [Schwern's] imagination” by enabling a bunch of modules automatically, like Modern::Perl, Time::Piece and autobox.

Then there was something about how being polyamorous is like Agile software development.

After the opening spiel came Richard Dice, the president of The Perl Foundation, gave us an update on TPF and Perl over the past year. From someone who doesn’t follow TPF it looks like financially it’s doing fairly well. It’s a $200,000 donation from Ian Hague in May 2008, it received a $50,000 donation from booking.com, and received another $70,000 from other donations. It has given out grants to help support Perl 6 development, to Nick Clark to release Perl 5.8.9, and to David Mitchell to release Perl 5.10.1.

In March 2009 The Parrot Foundation was spun off from TPF.

TPF gave out five grands in each of the second and third quarters of 2008, two in the fourth quarter, and one in the first quarter of 2009. It’s sponsoring nine Google Summer of Code projects, up from five last year.

Over the past year, Perl 5 has moved to git for version control. Strawberry Perl, for running Perl on Windows, was released. Various Perl groups were formed, such as the Japan Perl Association and the Enlightened Perl Organization.

There were many Perl-related events over the past year, from OSCON to workshops and hackathons around the world.

After Richard, Larry Wall gave a keynote about error messages and how he’s making the ones in Perl 6 chatty.

After Larry, Jerry Gay from The Parrot Foundation gave a talk, and after him came Yaakov, telling new YAPCers to go talk to people.

Well now you don’t have to! He’s made it available for free on-line!

Of course, if you want to support Perl programmers you should still buy his book.

Which transitions nicely into colour schemes. Contrast is essential. Contrast is essential. Contrast is essential. There are many tools that allow you to generate complementary colour schemes: use them. Keep in mind that about 10% of the population has some difficulty with colour perception, so don’t use colour as a tool for discriminating between different texts, use brightness instead (or maybe different fonts). There’s a tool at vischeck.com that one can use to view webpages under different types of colour impairment.

It’s essential to keep people interested, so every so often surprise your audience. Tell them a story, use examples, make comparisons, change your pace and style (but don’t change your pace just for the sake of it). And every so often, give your staircases landings to give people a chance to digest what they’re injesting. You can also use those landings to let people know that a new topic is coming up, to keep the audience oriented with the flow of the presentation.

“Just because you can doesn’t mean you should.” He’s talking about transitions. Make them smooth and elegant, and use the obvious transitions in special occasions, possibly when you come to a “landing.” And for the love of Pete, don’t use transitions between slides, either fade in or just switch to the next one. Don’t zoom in, don’t do cube rotation, don’t do any of that rot.

Don’t use architecture charts. I’ve been guilty of this, and nearly every presentation I’ve seen has one. “Show a chart, lose an entire audience.” If the audience needs to know it, put it in the notes. Ditto for graphs.

If you’re going to put your name and title on your slides (which isn’t a bad idea), make them subtle and ignorable so they don’t interfere with the information on your slides. Lose the logo from each slide, because putting the logo on is distracting and it makes you look like a salesperson. If you have to have a logo because of corporate policy, watermark it.

Moving on to presentation, make it look effortless. Make it look like you’re on TV so your audience doesn’t feel pity for you. That’s something that’s learned and comes with practice. Make it look easy, even if the actual material is difficult, because the audience wants it to look easy and not threatening. And be yourself, but be yourself-less. The presentation isn’t about you, it’s about your material. You’re a conduit for relaying the information to the audience.

“Don’t be paralyzed by the butterflies.” Use your nervousness and turn it into nervous energy. Rituals can help you focus, get you into the same groove. Rituals can be anything, but as long as it takes you back to the familiar it’s good, it’s less frightening. He recommends not ignoring the audience before doing your presentation, but if you talk with someone, talk about them instead of yourself.

Have a backup plan. Always carry backups of your material. Conway’s a little anal about backups (he says “sophisticated”), with five backups in five different locations, but it works for him. It may only save you once, but that’s enough.

Dress how your audience expects you to dress. If you’re a CEO, dress like a CEO. If you’re a hacker, dress like a hacker.

Don’t read your presentation. The audience is perfectly capable of reading the presentation, they don’t need it read for them. Unless you’re reading to 2-year olds, of course. To do this, make sure you rehearse your presentation, and rehearse it aloud. You can have a cheat sheet, but it’s not the script, it’s a list of points you are going to make.

Demonstrations beat descriptions, because it’s easier for the audience to get the point. Make sure that your demonstrations work, though, and make sure that you’ve got local copies of your demos, if they normally live on the Internet somewhere. Use IO::Prompt if at all possible.

Use a microphone, drink water, and make a pitstop before you start. Watch the time, and make sure you stick to your intended schedule.

For handling questions, have a policy. Don’t just take questions randomly, decide where and when and if you’ll take questions, and let the audience know about this policy. When you answer questions, repeat the question and try to paraphrase it. If you don’t know the answer, don’t fake it, say something clever instead like “I don’t know the answer,” or try to redirect the question to someone who might know the answer.

And try to do presentations fairly often. Even if they’re not formal presentations, there are many situations you can use to improve your skills. “The only way to speak better is to speak.”

[link to first half]

20 minutes in and the main point seems to be “be confident.” To be confident you need to know your material beforehand and know it well. If you don’t know it well then you’ll be up there giving a presentation, stumbling all over the place. You need to prepare your presentation and notes beforehand and practice it. You need to talk about what you care about.

You should take quite a bit of time to come up with your presentation. Conway suggested 10 hours of prep for each hour of presentation. 20:1 if you’ve got something relatively tough. He said for this three hour tutorial he took 80 hours to come up with it, and was still fiddling with it the night before.

Oh, and you need to be entertaining: “Entertaining always trumps informative.” Tell a story. Even if it’s made up, it’ll keep the audience interested and keep the presentation topic grounded in reality. Use metaphors, but use good metaphors: “I’ve never considered time as a non-renewable resource.”

Sidenote: This must be a good presentation because in a room full of geeks only a handful have their laptops out, and there are only two or three typing. End sidenote.

You should have five major points that the listener is going to take away with them. Humans don’t have a very big buffer so any more information than this will just leak out of their ears.

He’s currently making the point that presentation is more important than the information: “having great information on a shoddy-looking slide is actually worse than shoddy information on a great-looking slide.” I agree with his point, but for the most part I think he should be making the point that simplicity is more important than attacking the user with graphics. Actually, it looks like he is, but doing so in a bit of a roundabout fashion instead of outright saying it.

Yes, he is promoting simplicity. For presentation style, don’t use the Microsoft standard PowerPoint templates. “Steal from the cool kids: Apple, Bang & Olufsen, The Perl Journal… no substance, but lots of style.” For fonts, don’t use many. Give each font a meaning: italics for headings, classic serif for content, fixed width for code. Don’t use Papyrus (it’s overused) or Comic Sans (it looks amateurish). If you use images, use them meaningfully and in moderation, “like seasoning.” Ditto for animations, but use them even more sparingly: “if I can’t do it easily with the limited facilities in PowerPoint, it’s probably too complex to bother with.” Ditto for video, but use video even less often than animations.

We’re now at the half-way point. Time to get some coffee.

[link to second half]