Leap second, again

News: There will be another leap second on 23:59:60UTC 31-JUN-2012.

Nevertheless, I quitted supporting leap second on the time zone file at home last year on December 2011, which had been introduced for more than 12 years. Reason: I gain very little while I lose quite a lot by setting the system time based on TAI.

I think POSIX should have defined the timezone functions to handle leap seconds properly, by setting the TAI-based monotonically-increasing value to the local clock, and by correcting the difference between UTC and TAI with the timezone file. But POSIX doesn't. So TAI-based local time is not interchangeable. Most applications assume the value of time_t represents UTC without the leap seconds, which is not the case when the local clock is set based on TAI.

I suspect that many sysadmins are not really aware of the leap seconds, unfortunately, including those provide NTP services. I've experienced one of the upper-stratum NTP servers of my ISP did not synchronize on 1-JAN-2009, when the last leap second was added.

Very few language systems can explicitly handle leap seconds either. For example, I had to post a patch for Erlang for handling TAI. (Fixed on Erlang/OTP R13A and later.)

So you should be very careful on the de-synchronization of your machine's local clocks on June 30th or July 1st this year on 2012.

Useful links:

• The Future of Leap Seconds
• Leap Seconds by Markus Kuhn
• Communications ACM May 2011: The One-Second War
• The NTP Timescale and Leap Seconds
• D. J. Bernstein: UTC, TAI, and UNIX time

(Correction 8-JAN-2012: gmtime() was ambiguous. Corrected to time_t.)

Preparing FreeBSD Port of Erlang/OTP

Updating an Erlang/OTP Port (files under the Ports tree) for FreeBSD is not a complicated task, with the latest pieces of the Port files. You can find out the latest pieces at the FreeBSD CVSweb page for Erlang Port.

You need to do the following things:

• Updating the source file names
• Preparing the library list
• Preparing the man file list
• Removing/editing/adding patches
• Updating the checksum file
• Preparing the package list
• Test building
• References

Updating the source file names

The following parts of the main Makefile must be modified to specify the related source file names (in a unified diff-like format):

    -PORTVERSION=   r14b04
    +PORTVERSION=   r15b

    -DISTNAME=      otp_src_R14B04
    +DISTNAME=      otp_src_R15B

    -ERLANG_MAN=    otp_doc_man_R14B04.tar.gz
    +ERLANG_MAN=    otp_doc_man_R15B.tar.gz

    -ERLANG_DOCS=   otp_doc_html_R14B04.tar.gz
    +ERLANG_DOCS=   otp_doc_html_R15B.tar.gz

Preparing the library list

Makefile.lib must be supplied to provide all the version numbers of the OTP library modules.

To extract the version number, use the following shell command:

    (cd work/otp_src_VERSION; \
     find . -name vsn.mk -print | xargs grep VSN) \
     > OTP-vsnlist.txt

The content of OTP-vsnlist.txt will be something like this:

    ./erts/vsn.mk:VSN = 5.9
    ./erts/vsn.mk:SYSTEM_VSN = R15B
    ./lib/appmon/vsn.mk:APPMON_VSN = 2.1.14
    ./lib/asn1/vsn.mk:ASN1_VSN = 1.6.19
    ./lib/common_test/vsn.mk:COMMON_TEST_VSN = 1.6
    ./lib/compiler/vsn.mk:COMPILER_VSN = 4.8
    ./lib/cosEvent/vsn.mk:COSEVENT_VSN = 2.1.12
    ./lib/cosEventDomain/vsn.mk:COSEVENTDOMAIN_VSN = 1.1.12
    ./lib/cosFileTransfer/vsn.mk:COSFILETRANSFER_VSN = 1.1.13
    ./lib/cosNotification/vsn.mk:COSNOTIFICATION_VSN = 1.1.18
    [...]
    ./lib/wx/vsn.mk:WX_VSN = 0.99.1
    ./lib/xmerl/vsn.mk:XMERL_VSN = 1.3

You have to manually edit the Makefile.lib with the content of OTP-vsnlist.txt. The format of Makefile.lib is (beware that the last line has no backslash):

    ERTS_VSN=       5.9
    TOOLS_VSN=      2.6.6.6
    OTP_LIBS=       appmon-2.1.14 \
                    asn1-1.6.19 \
                    common_test-1.6 \
                    compiler-4.8 \
                    cosEvent-2.1.12 \
                    cosEventDomain-1.1.12 \
                    cosFileTransfer-1.1.13 \
                    cosNotification-1.1.18 \
                    [...]
                    wx-0.99.1 \
                    xmerl-1.3

The contents of Makefile.lib is a set of Makefile variables, so all the module names must be listed with the backslash character. (Note: ERTS_VSN is no longer needed, but I will include it here just FYI.)

Preparing the man file list

Makefile.man must be supplied to provide all the version numbers of the OTP library modules.

To extract the man filenames, use the following shell command:

    tar ztf /usr/ports/distfiles/erlang/otp_doc_man_VERSION.tar.gz | \
    sort | awk '{FS="/"; print $3;}' > OTP-manlist.txt

You have to manually edit the Makefile.man with the content of OTP-manlist.txt. For OTP R15B, Makefile.man only needs man sections 1, 3, 4, and 6, so man section 7 must not be included.

The content of OTP-manlist.txt will be something like this:

COPYRIGHT
PR.template
README

man1/
man1/ct_run.1
man1/diameter_compile.1
man1/epmd.1
man1/erl.1
man1/erl_call.1
man1/erlc.1
[...]
man7/SNMP-VIEW-BASED-ACM-MIB.7
man7/SNMPv2-MIB.7
man7/SNMPv2-TM.7
man7/STANDARD-MIB.7
man7/TRANSPORT-ADDRESS-MIB.7

You have to manually edit the Makefile.man with the content of OTP-manlist.txt. The format of Makefile.man is (beware that the last line of each section has no backslash):

    MAN1=   ct_run.1 \
            diameter_compile.1 \
            epmd.1 \
            erl.1 \
            erl_call.1 \
            erlc.1 \
            erlsrv.1 \
            escript.1 \
            run_erl.1 \
            snmpc.1 \
            [...]
            start_webtool.1 \
            werl.1

    MAN3=   CosEventChannelAdmin.3 \
            CosEventChannelAdmin_ConsumerAdmin.3 \
            [...]
            zlib.3 \
            zlib_stub.3

    MAN4=   app.4 \
            appup.4 \
            [...]
            relup.4 \
            script.4

    MAN6=   common_test.6 \
            crypto.6 \
            [...]
            stdlib.6 \
            test_server.6

The contents of Makefile.man is a set of Makefile variables, so all the man file names must be listed with the backslash character.

Removing/editing/adding patches

FreeBSD-specific patch files are provided under the Port directory files. You need to remove the obsolete patches first, by make patch. You should edit the existing ones and add the new ones as well.

The naming convention of the patch file is:

    patch-lib_et_src_et__gs__contents__viewer.erl

for

    lib/et/src/et_gs_contents_viewer.erl

(Conversion of / -> _ and _ -> __ is needed.)

Updating the checksum file

make makesum is the convenient way to update the distinfo checksum data.

Preparing the package list

New binary files are occasionally added in the OTP distributions. The pkg-plist file must be updated as well.

Test building

The building process will be like this (note the umask 022):

    # I use zsh, but use whatever you want
    # as root
    sudo zsh
    umask 022
    make fetch
    exit
    # as your own account
    umask 022
    make
    # as root
    sudo zsh
    umask 022
    make deinstall && make reinstall
    exit

(Note added 5-MAR-2012: I've posted a send-pr (ports/163711) to FreeBSD issue tracker system on 30-DEC-2011. I've made an Evernote page for the port tar.gz archive.)

References

• Erlang/OTP repository at GitHub
• FreeBSD Porter's Handbook

It's all about experience

It's not just the programming.
It's not just the music.
It's not just the message.

It's all about experience. And the desire for it.

People are looking for a new experience through technologies.

Concurrency will surely help giving a smooth experience; and even more good things.

(I will not quote Jimi Hendrix here)

Twitter in Japan after 3-11 or 11-MAR-2011

I'm really thinking about restricting my tweets to tech issues only.

Tweets in Japanese have been getting too political these days, especially after the 3-11 accidents and disasters. People are less restrictive on expressing op-ed tweets, and the tweets further invoke even more harsh exchanges of discussions without careful thoughts, often lead to blackmailing.

I do understand people in Japan including myself have legitimate fears against the possible nuclear and predicted earthquake/tsunami disasters. And I accept people have rights to openly discuss those matters. Twitter, however, is not a good place to have discussions with showing sufficient clarifications and enough pieces of evidence.

My technical interests for Erlang/OTP as of April 2011

This is a list of my technical interests for Erlang/OTP as of April 2011:

• Catching up the latest cryptographic algorithms, including hash functions, shared-key cryptography, public-key cryptography, pseudo random number generators (PRNGs), while making them efficient in parallel/concurrent execution environment;
• Delay tolerance network technologies which can handle days, weeks, months of message delivery delays;
• Fundamental support functions of IPv6, especially for Distributed Erlang and epmd, and cryptographic transport support with SSL/TLS;
• DNSSEC-compatible resolver implementation for inet_res module;
• Large binary heap handling of BEAM, and the programming methodologies for the faster NIF implementation, including the performance measurement;
• PRNGs with small context/internal-state size, e.g., Wichmann-Hill 1982/2006, LShift, XOR32;
• PRNGs with large context/internal-state size, e.g., SFMT;
• TAI or leap-second handling by NIF;
• Privilege separation techniques between BEAM instances (e.g., virtualization, FreeBSD Jail); and
• the effective usage and choice of port drivers, linked-in drivers, and NIFs

And projects I'm interested in (but not limited to):

• Disco Project, a MapReduce implementation
• ZeroMQ and other general queueing systems
• UBF, BERT-RPC, and other messaging formats
• CouchDB

And don't forget I'll be hosting the ACM Erlang Workshop 2011.

(This article is a revised version of http://heikou-konton.blogspot.com/2011/02/erlangotp.html (originally written in Japanese.)

Erlang Workshop 2011 will be held as scheduled

This is a short notice that Erlang Workshop 2011 aka The Tenth ACM SIGPLAN Erlang Workshop will be held as scheduled at National Center of Sciences, National Institute of Informatics, Tokyo, Japan, on Friday, September 23, 2011.

See my presentation slides at ErlLounge of Erlang Factory SF Bay 2011 for the further details (in PDF). I should emphasize that the distance between the troubled Fukushima nuclear power plants and Tokyo is more than 200 kilometers, and also that the venue building of the workshop will not be affected by the scheduled rolling blackouts.

I also suggest you to take a look at ICFP 2011 Web site for the details of the ICFP 2011 related events.

3-FEB-2011: IPv4 address pool in IANA is used up

As of 3 February 2011, the central pool of available IPv4 addresses managed by the Internet Assigned Numbers Authority (IANA) has been depleted. See ARIN's FAQ for the further details. Since July 1994 when IPv6 was first recommended by the IETF, Internet engineers are periodically warning people that IPv4 address space is finite. And the day has come. Of course this is not the end of the Internet. We need to consider, however, some new issues:

• Internet is no longer a unified network of single protocol; it will become a multi-protocol network of IPv4 and IPv6 shortly.
• While quite a new technologies have been developed and are under deployment for the migration to IPv6 (i.e., dumping the old technology of IPv4 and replace it by IPv6), we will still have to endure the fact that IPv4 will be staying around for another few years, or even more than 10 years.
• Promoting IPv6 is essential. I think, however, maintaining the inter-connectivity of the two networks and the services provided there from both networks is even more critical. We don't want two split networks; we still want one unified Internet, don't we?