Showing posts with label FreeBSD. Show all posts
Showing posts with label FreeBSD. Show all posts

Thursday, April 24, 2014

My new startup: Kenji Rikitake Professional Engineer's Office

This is an announcement of my new startup business.

I've started my own business, as a Japanese government-licensed Professional Engineer of Information Technology (in Japanese it's called Gijyutsushi), since April 21, 2014. It's called Kenji Rikitake Professional Engineer's Office (KRPEO).

KRPEO provides the consulting and engineering services on information technology in general, including but not limited to: information and network security, the design, deployment, implementation and performance tuning of Erlang/OTP, FreeBSD, and Riak. KRPEO will provide all services in both Japanese and English.

I had been looking for a full-time job since October 2013, and I made a conclusion that the job market near Osaka was mostly for energy-consuming legacy programming tasks due to the fact that most of the high-end software engineering companies were only located in Tokyo. Most of the employers in Japan do not accept remote teleworking due to the cultural and legal reasons either. So I had to lean out from the traditional corporate culture of Japan, for a more sustainable business model, making myself, my family, and my customers happy at the same time. I understand this is a hard challenge, but I will definitely take it.

My involvement in the open-source developer and network operator communities in Japan and the world will remain the same and unchanged.

The English announcement site for KRPEO is at http://www.k2r.org/krpeo/. The Japanese site URL is: http://www.k2r.org/gijyutsushi/.

Update 28-APR-2014: add the URL of the English site, correct the URL of the Japanese site.

Monday, March 5, 2012

Erlang VM memory size and ERL_MAX_PORTS

Erlang VM has the ports to communicate with other programs and objects (files, drivers, etc.) The ports are seen as the Erlang processes. The OS environment variable ERL_MAX_PORTS sets the default value of maximum number of ports. (See the manual for the function erlang:open_port/2 for the further details.)

I discovered by accident that Erlang VM of R15B on FreeBSD/amd64 9.0 in my test environment sets the value of maximum file descriptors assigned to the OS process as the default value for ERL_MAX_PORTS. This means ERL_MAX_PORTS is set to 800000, far larger than 1024, specified in the manual. This excessive value of the ERL_MAX_PORTS leads into unnecessary memory consumption of the Erlang VM. An easy way to test this is to execute the following code on the shell:

env ERL_MAX_PORTS=[given value] erl -noinput -eval 'io:format("~p~n",[erlang:memory(system)]).' -s erlang halt

How memory consumption changes just after the Erlang VM startup, as {ERL_MAX_PORTS, memory size in bytes}: {1024, ~65M}, {102400, ~124M}, {800000, ~543M}.

So I decided to set the ERL_MAX_PORTS for my home Yaws server to 4096. This significantly reduced the memory consumption.

I suggest you to tune the value of ERL_MAX_PORTS to optimize the number of concurrently opening ports of an Erlang VM, especially when you run a server on a memory-restricted host.

Update 11-MAR-2013: on R16B, the amount of memory consumed by the ports will be significantly reduced at the Erlang VM startup.

References

Wednesday, January 4, 2012

Preparing FreeBSD Port of Erlang/OTP

Updating an Erlang/OTP Port (files under the Ports tree) for FreeBSD is not a complicated task, with the latest pieces of the Port files. You can find out the latest pieces at the FreeBSD CVSweb page for Erlang Port.

You need to do the following things:

Updating the source file names

The following parts of the main Makefile must be modified to specify the related source file names (in a unified diff-like format):

    -PORTVERSION=   r14b04
    +PORTVERSION=   r15b

    -DISTNAME=      otp_src_R14B04
    +DISTNAME=      otp_src_R15B

    -ERLANG_MAN=    otp_doc_man_R14B04.tar.gz
    +ERLANG_MAN=    otp_doc_man_R15B.tar.gz

    -ERLANG_DOCS=   otp_doc_html_R14B04.tar.gz
    +ERLANG_DOCS=   otp_doc_html_R15B.tar.gz

Preparing the library list

Makefile.lib must be supplied to provide all the version numbers of the OTP library modules.

To extract the version number, use the following shell command:

    (cd work/otp_src_VERSION; \
     find . -name vsn.mk -print | xargs grep VSN) \
     > OTP-vsnlist.txt

The content of OTP-vsnlist.txt will be something like this:

    ./erts/vsn.mk:VSN = 5.9
    ./erts/vsn.mk:SYSTEM_VSN = R15B
    ./lib/appmon/vsn.mk:APPMON_VSN = 2.1.14
    ./lib/asn1/vsn.mk:ASN1_VSN = 1.6.19
    ./lib/common_test/vsn.mk:COMMON_TEST_VSN = 1.6
    ./lib/compiler/vsn.mk:COMPILER_VSN = 4.8
    ./lib/cosEvent/vsn.mk:COSEVENT_VSN = 2.1.12
    ./lib/cosEventDomain/vsn.mk:COSEVENTDOMAIN_VSN = 1.1.12
    ./lib/cosFileTransfer/vsn.mk:COSFILETRANSFER_VSN = 1.1.13
    ./lib/cosNotification/vsn.mk:COSNOTIFICATION_VSN = 1.1.18
    [...]
    ./lib/wx/vsn.mk:WX_VSN = 0.99.1
    ./lib/xmerl/vsn.mk:XMERL_VSN = 1.3

You have to manually edit the Makefile.lib with the content of OTP-vsnlist.txt. The format of Makefile.lib is (beware that the last line has no backslash):

    ERTS_VSN=       5.9
    TOOLS_VSN=      2.6.6.6
    OTP_LIBS=       appmon-2.1.14 \
                    asn1-1.6.19 \
                    common_test-1.6 \
                    compiler-4.8 \
                    cosEvent-2.1.12 \
                    cosEventDomain-1.1.12 \
                    cosFileTransfer-1.1.13 \
                    cosNotification-1.1.18 \
                    [...]
                    wx-0.99.1 \
                    xmerl-1.3

The contents of Makefile.lib is a set of Makefile variables, so all the module names must be listed with the backslash character. (Note: ERTS_VSN is no longer needed, but I will include it here just FYI.)

Preparing the man file list

Makefile.man must be supplied to provide all the version numbers of the OTP library modules.

To extract the man filenames, use the following shell command:

    tar ztf /usr/ports/distfiles/erlang/otp_doc_man_VERSION.tar.gz | \
    sort | awk '{FS="/"; print $3;}' > OTP-manlist.txt

You have to manually edit the Makefile.man with the content of OTP-manlist.txt. For OTP R15B, Makefile.man only needs man sections 1, 3, 4, and 6, so man section 7 must not be included.

The content of OTP-manlist.txt will be something like this:

COPYRIGHT
PR.template
README

man1/
man1/ct_run.1
man1/diameter_compile.1
man1/epmd.1
man1/erl.1
man1/erl_call.1
man1/erlc.1
[...]
man7/SNMP-VIEW-BASED-ACM-MIB.7
man7/SNMPv2-MIB.7
man7/SNMPv2-TM.7
man7/STANDARD-MIB.7
man7/TRANSPORT-ADDRESS-MIB.7

You have to manually edit the Makefile.man with the content of OTP-manlist.txt. The format of Makefile.man is (beware that the last line of each section has no backslash):

    MAN1=   ct_run.1 \
            diameter_compile.1 \
            epmd.1 \
            erl.1 \
            erl_call.1 \
            erlc.1 \
            erlsrv.1 \
            escript.1 \
            run_erl.1 \
            snmpc.1 \
            [...]
            start_webtool.1 \
            werl.1

    MAN3=   CosEventChannelAdmin.3 \
            CosEventChannelAdmin_ConsumerAdmin.3 \
            [...]
            zlib.3 \
            zlib_stub.3

    MAN4=   app.4 \
            appup.4 \
            [...]
            relup.4 \
            script.4

    MAN6=   common_test.6 \
            crypto.6 \
            [...]
            stdlib.6 \
            test_server.6

The contents of Makefile.man is a set of Makefile variables, so all the man file names must be listed with the backslash character.

Removing/editing/adding patches

FreeBSD-specific patch files are provided under the Port directory files. You need to remove the obsolete patches first, by make patch. You should edit the existing ones and add the new ones as well.

The naming convention of the patch file is:

    patch-lib_et_src_et__gs__contents__viewer.erl

for

    lib/et/src/et_gs_contents_viewer.erl

(Conversion of / -> _ and _ -> __ is needed.)

Updating the checksum file

make makesum is the convenient way to update the distinfo checksum data.

Preparing the package list

New binary files are occasionally added in the OTP distributions. The pkg-plist file must be updated as well.

Test building

The building process will be like this (note the umask 022):

    # I use zsh, but use whatever you want
    # as root
    sudo zsh
    umask 022
    make fetch
    exit
    # as your own account
    umask 022
    make
    # as root
    sudo zsh
    umask 022
    make deinstall && make reinstall
    exit

(Note added 5-MAR-2012: I've posted a send-pr (ports/163711) to FreeBSD issue tracker system on 30-DEC-2011. I've made an Evernote page for the port tar.gz archive.)

References

Saturday, May 17, 2008

Installing FreeBSD 6.3 and 7.0 to Panasonic CF-R4

I had difficulties installing FreeBSD 6.3-RELEASE and 7.0-RELEASE to Panasonic Note PC CF-R4. This machine has been known for having an incompatible BIOS which prevents FreeBSD BTX 1.01 or earlier to boot.

Recent FreeBSD 8.0-CURRENT snapshot ISO images are with BTX 1.02, which allows CF-R4 booting from USB CD-ROM drives. So I tested the following procedure for the 6.3 and 7.0 releases and the both releases were successfully installed:

  • boot from a CD-R(W) with 8.0-CURRENT-200805-i386-bootonly.iso (or later)
  • exit the boot prompt screen to the loader prompt (by pressing 6) before the 8.0 kernel boots itself
  • enter unload to clear the loaded (8.0) kernel
  • replace the CD-R(W) in the drive to the target installation image (e.g., Disc 1 of 6.3/7.0-RELEASE)
  • enter boot-conf to load the target kernel and install

I think this is the compatibility problem of BTX. I also think this is so far the easiest way to boot from the USB CD-ROM drive for CF-R4, because generating ISO images for previous releases with the compatible BTX loader will be a big hassle. You may give it a try if you want to boot from a USB flash memory.