Concurrently Chaotic

Kernel module functions for IPv6 on Erlang/OTP R15B

2012-03-02T12:00:00.000+09:00

I've been reading the source code ofErlang/OTP R15Bto find out how it handles the IPv6-related functions.

Erlang VM has the most functions embedded into the inet_drv linked-in driver,so programmers don't really have to worry about many things except for the address tuple formatdifference between IPv4 ({192, 168, 0, 1}) and IPv6 ({8193,3512,4660,22136,51966,47806,57005,48879} for2001:db8:1234:5678:cafe:babe:dead:beef).You can find a lot of gems by reading the kernel module source code.

Here is a list of notable functions inside the kernel module for IPv6 handling on Erlang/OTP R15B.I suggest you to read these functions before writing your own ones.

(Note: this is a work-in-progress article ofmy Erlang Factory SF Bay Area 2012 presentation.

%%% A partial list of IPv6 keywords in Erlang/OTP R15B source code

%%% atom "inet6" is an important keyword

inet_parse:address/1
inet_parse:ipv6_address/1
inet_parse:ipv6strict_address/1 % review RFC4291
inet_parse:ntoa/1

inet_parse:hosts/2 % internal, now ignoring "%if" suffix

-type inet:ip6_address()

inet:getifaddrs/0
inet:getifaddrs/1
inet:gethostbyaddr/1
inet:getaddrs/2 % second arg: inet6
inet:gethostbyname_string/2 % second arg: inet6

inet:i/0
inet:i/1

%%% interesting module results
%%% (no significant difference other than inet/inet6 flags):
%%%     diff --unified inet_tcp.erl inet6_tcp.erl
%%%     diff --unified inet_udp.erl inet6_udp.erl
%%%     diff --unified inet_sctp.erl inet6_sctp.erl
%%%     diff --unified inet_tcp_dist.erl inet6_tcp_dist.erl

inet_config:set_hostname/0 % internal, dependent on inet_udp:open/2

%%% inet_db.erl has tcp/udp/sctp module config params
%%% for the gen_server callbacks

inet_db:set_inet6/1
inet_db:dn_ip6_int/8 % internal, for 8 elements of an IPv6 address tuple
inet_db:make_hostent(Name, Addrs, Aliases, ?S_AAAA)
inet_db:handle_call({add_host, IPv6address, Names})
inet_db:handle_call({del_host, IPv6address})

inet_dns.erl: ?S_AAAA related lines

inet_dns.hrl: --define(T_AAAA), --define(S_AAAA)

inet_int.hrl: --define(INET_AF_INET6), --define(SCTP_FLAG_PMTUD_*),
              --define(ip6(A,B,C,D,E,F,G,H)

-type inet_res:res_option({inet6, boolean()})
-type inet_res:dns_data() % inet:ip6_address()

inet_res:gethostbyaddr/1
inet_res:gethostbyaddr_tm/2
inet_res:gethostbyname/1
inet_res:gethostbyname_tm/3
inet_res:udp_open/2 % internal (record #sock)
inet_res:udp_connect/2 % internal (record #sock)

% There will be more...

Small pieces of happiness that I pursue 2012-02-25T12:00:00.000+09:00 The small pieces of happiness I've been pursuing since 1992 are as follows: Making money by doing what I want and by minimizing doing what I don't want to do; Living in a house of my own control with my partner without interference; Working at a place where I want and at a place where I can minimize the mental stress; and Working at a period of time in the most productive way, including taking enough time to rest. I feel I've been devoting myself to pursue this small happiness at my best effort. And I'm happy with it. Leap second, again 2012-01-08T00:00:00.000+09:00 News: There will be another leap second on 23:59:60UTC 31-JUN-2012. Nevertheless, I quitted supporting leap second on the time zone file at home last year on December 2011,which had been introduced for more than 12 years. Reason: I gain very little while I lose quite a lot by setting the system time based on TAI. I think POSIX should have defined the timezone functions to handle leap seconds properly, by setting the TAI-based monotonically-increasing value to the local clock, and by correcting the difference between UTC and TAI with the timezone file. But POSIX doesn't. So TAI-based local time is not interchangeable. Most applications assume the value of time_t represents UTC without the leap seconds, which is not the case when the local clock is set based on TAI. I suspect that many sysadmins are not really aware of the leap seconds, unfortunately, including those provide NTP services. I've experienced one of the upper-stratum NTP servers of my ISP did not synchronize on 1-JAN-2009, when the last leap second was added. Very few language systems can explicitly handle leap seconds either. For example,I had to post a patch for Erlang for handling TAI. (Fixed on Erlang/OTP R13A and later.) So you should be very careful on the de-synchronization of your machine's local clocks on June 30th or July 1st this year on 2012. Useful links: The Future of Leap Seconds Leap Seconds by Markus Kuhn Communications ACM May 2011: The One-Second War The NTP Timescale and Leap Seconds D. J. Bernstein: UTC, TAI, and UNIX time (Correction 8-JAN-2012: gmtime() was ambiguous. Corrected to time_t.) Preparing FreeBSD Port of Erlang/OTP 2012-01-04T00:00:00.000+09:00 Updating an Erlang/OTP Port (files under the Ports tree) for FreeBSD is not a complicated task,with the latest pieces of the Port files.You can find out the latest pieces atthe FreeBSD CVSweb page for Erlang Port. You need to do the following things: Updating the source file names Preparing the library list Preparing the man file list Removing/editing/adding patches Updating the checksum file Preparing the package list Test building References Updating the source file names The following parts of the main Makefile must be modifiedto specify the related source file names(in a unified diff-like format): -PORTVERSION= r14b04 +PORTVERSION= r15b -DISTNAME= otp_src_R14B04 +DISTNAME= otp_src_R15B -ERLANG_MAN= otp_doc_man_R14B04.tar.gz +ERLANG_MAN= otp_doc_man_R15B.tar.gz -ERLANG_DOCS= otp_doc_html_R14B04.tar.gz +ERLANG_DOCS= otp_doc_html_R15B.tar.gz Preparing the library list Makefile.lib must be supplied to provideall the version numbers of the OTP library modules. To extract the version number, use the following shell command: (cd work/otp_src_VERSION; \ find . -name vsn.mk -print | xargs grep VSN) \ > OTP-vsnlist.txt The content of OTP-vsnlist.txt will be something like this: ./erts/vsn.mk:VSN = 5.9 ./erts/vsn.mk:SYSTEM_VSN = R15B ./lib/appmon/vsn.mk:APPMON_VSN = 2.1.14 ./lib/asn1/vsn.mk:ASN1_VSN = 1.6.19 ./lib/common_test/vsn.mk:COMMON_TEST_VSN = 1.6 ./lib/compiler/vsn.mk:COMPILER_VSN = 4.8 ./lib/cosEvent/vsn.mk:COSEVENT_VSN = 2.1.12 ./lib/cosEventDomain/vsn.mk:COSEVENTDOMAIN_VSN = 1.1.12 ./lib/cosFileTransfer/vsn.mk:COSFILETRANSFER_VSN = 1.1.13 ./lib/cosNotification/vsn.mk:COSNOTIFICATION_VSN = 1.1.18 [...] ./lib/wx/vsn.mk:WX_VSN = 0.99.1 ./lib/xmerl/vsn.mk:XMERL_VSN = 1.3 You have to manually edit the Makefile.libwith the content of OTP-vsnlist.txt.The format of Makefile.lib is(beware that the last line has no backslash): ERTS_VSN= 5.9 TOOLS_VSN= 2.6.6.6 OTP_LIBS= appmon-2.1.14 \ asn1-1.6.19 \ common_test-1.6 \ compiler-4.8 \ cosEvent-2.1.12 \ cosEventDomain-1.1.12 \ cosFileTransfer-1.1.13 \ cosNotification-1.1.18 \ [...] wx-0.99.1 \ xmerl-1.3 The contents of Makefile.lib is a set of Makefile variables,so all the module names must be listed with the backslash character.(Note: ERTS_VSN is no longer needed, but I will include ithere just FYI.) Preparing the man file list Makefile.man must be supplied to provideall the version numbers of the OTP library modules. To extract the man filenames, use the following shell command: tar ztf /usr/ports/distfiles/erlang/otp_doc_man_VERSION.tar.gz | \ sort | awk '{FS="/"; print $3;}' > OTP-manlist.txt You have to manually edit the Makefile.manwith the content of OTP-manlist.txt.For OTP R15B, Makefile.man only needs man sections 1, 3, 4, and 6,so man section 7 must not be included. The content of OTP-manlist.txt will be something like this: COPYRIGHT PR.template README man1/ man1/ct_run.1 man1/diameter_compile.1 man1/epmd.1 man1/erl.1 man1/erl_call.1 man1/erlc.1 [...] man7/SNMP-VIEW-BASED-ACM-MIB.7 man7/SNMPv2-MIB.7 man7/SNMPv2-TM.7 man7/STANDARD-MIB.7 man7/TRANSPORT-ADDRESS-MIB.7 You have to manually edit the Makefile.manwith the content of OTP-manlist.txt.The format of Makefile.man is(beware that the last line of each section has no backslash): MAN1= ct_run.1 \ diameter_compile.1 \ epmd.1 \ erl.1 \ erl_call.1 \ erlc.1 \ erlsrv.1 \ escript.1 \ run_erl.1 \ snmpc.1 \ [...] start_webtool.1 \ werl.1 MAN3= CosEventChannelAdmin.3 \ CosEventChannelAdmin_ConsumerAdmin.3 \ [...] zlib.3 \ zlib_stub.3 MAN4= app.4 \ appup.4 \ [...] relup.4 \ script.4 MAN6= common_test.6 \ crypto.6 \ [...] stdlib.6 \ test_server.6 The contents of Makefile.man is a set of Makefile variables,so all the man file names must be listed with the backslash character. Removing/editing/adding patches FreeBSD-specific patch files are provided under the Port directoryfiles. You need to remove the obsolete patches first, by make patch.You should edit the existing ones and add the new ones as well. The naming convention of the patch file is: patch-lib_et_src_et__gs__contents__viewer.erl for lib/et/src/et_gs_contents_viewer.erl (Conversion of / -> _ and _ -> __ is needed.) Updating the checksum file make makesum is the convenient way to update the distinfo checksum data. Preparing the package list New binary files are occasionally added in the OTP distributions.The pkg-plist file must be updated as well. Test building The building process will be like this(note the umask 022): # I use zsh, but use whatever you want # as root sudo zsh umask 022 make fetch exit # as your own account umask 022 make # as root sudo zsh umask 022 make deinstall && make reinstall exit References Erlang/OTP repository at GitHub FreeBSD Porter's Handbook It's all about experience 2011-08-19T23:24:00.000+09:00 It's not just the programming. It's not just the music. It's not just the message. It's all about experience. And the desire for it. People are looking for a new experience through technologies. Concurrency will surely help giving a smooth experience; and even more good things. (I will not quote Jimi Hendrix here) Twitter in Japan after 3-11 or 11-MAR-2011 2011-05-08T00:00:00.005+09:00 I'm really thinking about restricting my tweets to tech issues only. Tweets in Japanese have been getting too political these days, especially after the 3-11 accidents and disasters. People are less restrictive on expressing op-ed tweets, and the tweets further invoke even more harsh exchanges of discussions without careful thoughts, often lead to blackmailing. I do understand people in Japan including myself have legitimate fears against the possible nuclear and predicted earthquake/tsunami disasters. And I accept people have rights to openly discuss those matters. Twitter, however, is not a good place to have discussions with showing sufficient clarifications and enough pieces of evidence. My technical interests for Erlang/OTP as of April 2011 2011-05-07T13:51:00.000+09:00 This is a list of my technical interests for Erlang/OTP as of April 2011: Catching up the latest cryptographic algorithms, including hash functions,shared-key cryptography, public-key cryptography, pseudo random number generators (PRNGs),while making them efficient in parallel/concurrent execution environment; Delay tolerance network technologies which can handle days, weeks, months ofmessage delivery delays; Fundamental support functions of IPv6, especially for Distributed Erlang and epmd, andcryptographic transport support with SSL/TLS; DNSSEC-compatible resolver implementation for inet_res module; Large binary heap handling of BEAM, and the programming methodologiesfor the faster NIF implementation, including the performance measurement; PRNGs with small context/internal-state size, e.g., Wichmann-Hill 1982/2006, LShift, XOR32; PRNGs with large context/internal-state size, e.g., SFMT; TAI or leap-second handling by NIF; Privilege separation techniques between BEAM instances (e.g., virtualization, FreeBSD Jail); and the effective usage and choice of port drivers, linked-in drivers, and NIFs And projects I'm interested in (but not limited to): Disco Project, a MapReduce implementation ZeroMQ and other general queueing systems UBF, BERT-RPC, and other messaging formats CouchDB And don't forget I'll be hosting the ACM Erlang Workshop 2011. (This article is a revised version of http://heikou-konton.blogspot.com/2011/02/erlangotp.html (originally written in Japanese.) Erlang Workshop 2011 will be held as scheduled 2011-03-31T15:45:00.001+09:00 This is a short notice that Erlang Workshop 2011 aka The Tenth ACM SIGPLAN Erlang Workshop will be held as scheduled at National Center of Sciences, National Institute of Informatics, Tokyo, Japan, on Friday, September 23, 2011. See my presentation slides at ErlLounge of Erlang Factory SF Bay 2011 for the further details (in PDF). I should emphasize that the distance between the troubled Fukushima nuclear power plants and Tokyo is more than 200 kilometers, and also that the venue building of the workshop will not be affected by the scheduled rolling blackouts. I also suggest you to take a look at ICFP 2011 Web site for the details of the ICFP 2011 related events. 3-FEB-2011: IPv4 address pool in IANA is used up 2011-02-04T12:36:00.003+09:00 As of 3 February 2011, the central pool of available IPv4 addresses managed by the Internet Assigned Numbers Authority (IANA) has been depleted. See ARIN's FAQ for the further details.Since July 1994 when IPv6 was first recommended by the IETF, Internet engineers are periodically warning people that IPv4 address space is finite. And the day has come.Of course this is not the end of the Internet. We need to consider, however, some new issues:Internet is no longer a unified network of single protocol; it will become a multi-protocol network of IPv4 and IPv6 shortly. While quite a new technologies have been developed and are under deployment for the migration to IPv6 (i.e., dumping the old technology of IPv4 and replace it by IPv6), we will still have to endure the fact that IPv4 will be staying around for another few years, or even more than 10 years. Promoting IPv6 is essential. I think, however, maintaining the inter-connectivity of the two networks and the services provided there from both networks is even more critical. We don't want two split networks; we still want one unified Internet, don't we? Failing to import - now solved 2011-01-31T00:39:00.005+09:00 I am trying to import some old articles from one of my old Blogger.com exported file, still failing. Stay tuned. Update: now complete after tweaking the <author> section of the XML from the blogger.com exported file. Whew. Contents from old Concurently Chaotic have been moved. And the contents from my old blog jj1bdx: journal of technology have also been merged. Cloudsourcing from home 2011-01-30T12:06:00.003+09:00 Managing the information and services which I provide has been apersistent source of headache since I set up my own Web server at homein 1997. So far I've been successfully managing the contents, such as: External authoritative DNS (though you still need to maintain the internal authoritative DNS and the cache servers) Mail receiving servers (or more formally called the message transfer agents or MTAs) and the complex forwarding schemes for my family, namely me and Kyoko Static web servers solely for the public contents, including this blog I still have to pursue moving out the servers from home to the externalsites, however, because managing new protocols from home such as IPv6and other new applications is increasingly getting difficult, due to theconstraints of available bandwidth, IP address space, and my own time. Frankly speaking, I do not want to buy any more computer at home. Moreaccurately, I do not want to install and maintain operating systems andthe application software at home unless I really have to. Things havealready been getting too complex, and I need to offset or outsource thecomplexity to external sites and services. The problem I'm facing is that cloudsourcing or moving out servicesprovided at my home servers adds many levels of indirection and layersof things to consider: availability and redundancy issues, associationof the services, how and where to consistently collect the backup data,and the contingency plan and the recovery procedure in case of servicedisruption. I know the cloudsourcing issue is a matter of business reengineeringprocess, and to redesign the workflows of the various activities, in myfamily. It's not only about the computer systems. We need to changeour way of living. And I finally started to learn what kind of things you can do on GoogleApps, Google AppEngine, and Amazon Web Services. My new challenge in 2010: security issues at university network systems 2010-12-31T00:00:00.004+09:00 Since April 2010, I've been working on the network security managementof Kyoto University. There are many things to consider. I've alreadypublished a report in PDF about the status and issues. The fundamental principle which always haunts me is that adding securityto a not-so-secure system is far more difficult than making thesecurity component built-in into the system. In a large organization,you cannot change the policies and configuration of the running systemsovernight; you need to negotiate with a lot of stakeholders to reach aconsensus or an agreement. Consolidation takes a lot of time,especially for a well-established system such as the email servers. Another difficult issue to handle is how not to restrict users fromthe legitimate use of the campus systems. University is anorganization of research and education. While leaving a systemvulnerable for the known attack vectors is not an option anymore,allowing failures for learning from the try-and-error process should notbe prohibited. Without a firm security policy, you cannot really decidewhat is right or wrong. Catching up with new technologies is also a tough requirement to meetfor a large network system. The global IPv4 address space will be usedup in next year 2011, so introducing IPv6 is a must, though not urgent;and the organizational DNS subsystems should be DNSSEC-ready, for boththe cache resolvers and authoritative servers. And I have to be able toexplain those changes will surely benefit the users and are worth payingfor. I'm sure I'm going to face a lot of existing and new problems on nextyear 2011. And I'm hoping I will be able to solve at least some of them. Writing for job seeking 2010-03-14T00:00:00.001+09:00 For the past few months, my top priority issue has been searching for anew job. My current (as of March 2010) job contract will terminate byMarch 31, 2010. Looking for a job itself is very stressful, and youneed to focus on writing a lot of documents, including your CV, coverletters, etc. In this article I will introduce a few Web sites whichmight help job seekers in general. Ask A Manager, a blogwritten by a real human resource professional, has been particularlyhelpful during my job searching period. The articles on the blogsuggest me that corporate and organizational cultures in Japan and theUSA are much more similar than different with each other. I think thepsychology of hiring and getting hired is highly universal. For writing a CV, Dave Levin'sjob application page has a set of useful examples, especially forthose searching for IT research/academic positions, with the statementsof research and teaching. His energetic, upbeat and rational writingstyle, while not getting overly emotional, will help you find how tomake your CV fit for the job openings. (Note: of course you need tochange the details of your CV for each job you apply for.) Getting the job contract terminated, laid off, or even fired, is itselfa tedious and tough experience. I've read a few books for helping myself: Hugh MacLeod's IgnoreEverybody: this book and his comic strips remind me of the hardcold facts that creative people must endure loneliness, and that theyhave to earn money before doing what they want to do. MarthaI. Finney's Rebound iswhat ACM first suggested me for careerdevelopment. The good thing about this book is that it starts fromdescribing how to cope with the traumatic and angry feelings when youfirst find out you're getting pushed out of a job. ChadFowler's The Passionate Programmer, formerly called My JobWent to India, describes how a software developer should cope withthe change of trends and other social environmental factors. I highlyrecommend this book as a list of checkpoints for sorting out yourexperience and finding out what you need to do for doing what you reallywant to do. And I suggest you to think carefully about how you expose yourselfthrough blogs, Twitter, Facebook and other social media, during thetransition period of your job. Your articles or written pieces on the Internet are public and not private;anAsk A Manager article about how managers feel about blogging of theiremployees represents a practical view of pros and cons of bloggingin a plain English. I also suggest you to make your past blogs private, when youfeel awkward about what you have written there, even slightly. Then youcan reopen them after editing and removing the articles before you getembarrassed. Persistent Erlang processes or process pairs 2009-12-30T00:00:00.002+09:00 Erlang'sprocesses are the minimal unit of execution of Erlang BEAM virtualmachine. Each process has its own ID, and can send/receivemessages, register the name to the BEAM it's running, and can link withanother process for error handling and monitoring. It even has its ownprocess dictionary. I've been thinking about a question for a few days: can you make acomputer holding a process for more than 100 years? Theword process here does not necessarily have to be the Erlangone, but the Erlang process will be a good candidate because the workingenvironment it has to carry around with it is minimal and much smallerthan that of UNIX process. Keeping a process alive for a long time would not be possible if aprocess is confined to a single machine; the machine's failure means theimmediate death of the process. So the process should be able to movearound between multiple machines and make its clone on its own. Usingshared memory should be avoided as possible. Realizing thesecharacteristics with Erlang is less difficult than in other computerlanguage systems. I discovered the idea of having a persistent computer process isactually not my original. Google search engine tell me that Jim Grayhas already published a technical report when he was in Tandem in 1985(PDF copy from HP Labs) (a scanned text file of the report) with an idea of persistentprocess-pairs, as a part of his model of transactions forsimple fault-tolerant execution. In Gray's report, he describes a much smarter approach of making twopersistent processes a pair to realize the persistency. If one of thepair fails, another one will show up and take over the actions of thefailed one. This idea is much wiser than trying to keep a singleprocess alive. So now I find a way to realize persistent processes; next I need tolearn how to implement them. It'll be a part of my new year'sresolution for year 2010. A happy new year to you all the readers. DNS operation is utterly neglected by many people 2009-12-21T00:00:00.003+09:00 Twitter outage viaDNS hijacking showed another case of common symptom: DNSoperation is simply neglected by people doing business on the Internet. I was doing research on DNS transport security from 2002 to 2008. Oneof the reason I quit focusing on the research was that most, if not all,of the DNS problems are caused by operation failures, not necessarilydue to technical deficiency of the DNS protocols and systems. In short,it's too political and social to do the technological experiments overDNS. I still think DNS transport protocol issues are critical for stableInternet operation. But solving those issues does not help recoveringhuman errors, such as lame delegation (missing link) between the domainname hierarchy. And stable operation of DNS systems is very difficultto maintain without stable hardware, software, networks, and operators. I notice many small companies (especially in Japan) keep theirauthoritative servers inside their office, which is not good from thestability point of view. Actually, for many small Internet sites,including mine, not so many DNS zone records have to be exposed to thepublic. So I've already outsourced the DNS authoritative servers, whileI periodically watch whether those servers do the right thing. DNS is by definition a distributed system; and the management standardis much lower than what people (and even Internet engineers) believe.For the further details of how DNS is not well-managed, Isuggest you to read a more detailed commentary on howimportant DNS is as an asset, by Danny McPherson of Arbor Networks. Bruce Schneier's speech at IWSEC2009 2009-12-06T00:00:00.003+09:00 I had a chance to meet BruceSchneier face-to-face for the first time, when I attended hisinvited talk session at IWSEC2009conference in Toyama, Japan, on October 28, 2009. I once worked for translating Schneier's book Email Security (published in 1995, which is now declared outdated by him) intoJapanese. At that time he was a technologist on cryptography. Thekeynote speech in Toyama showed, however, that he was rather interestedin psychology and human behavior, which is not necessarily logicallypredictable and often considered errorneous from technological points ofview. While I read a few people who apparently tweeted Schneier's speech wasboring, I found his speech on the psycology of security ratherrefreshing and interesting. Maybe that's because I've been frequentlydisillusioned by how technological solutions often backfire. Of courseit's not about the details in cryptography or other security protocolswhich are the primary topics of IWSEC so that might have been boring forthe majority of the participants. I won't go into the details of Schneier's speech, because most of theindividual topics are frequently coveredin his blog. Let me writeabout one of the things intrigued me the most; it was about the riskheuristics. People are risk-aversed or trying to have sure gain. And atthe same time, they prefer probabilistic loss or risk-takingbehavior when they have possibilities of losing something. Withthis heurisric way of thinking, people usually don't want to pay forhaving less risky life, and this is exactly one of the reasons whysecurity products don't make good sales. After the speech, I asked him why he converted from pure technologist torather a scientist of broader topics including psycology and sociology.Unfortunately I didn't get a definitive answer on what made him so; heonly emphasized the sociological aspects of security were equallyimportant and critical as the technological ones. Maybe I could findthe answer in one of his books; especially if the reason is a highlypersonal one, which no one will ever know. Erlang and Github 2009-12-05T00:00:00.002+09:00 Erlang/OTP is now officiallymaintained under the Githubrepository, since the release R13B03. I think this is a milestonefor the language, because the Ericsson development team finally decidedto show the interim results of what they are doing for the time being. One of the characteristics I like about Erlang is that the languagespecification and libraries have been maintained by a single entitycalled Ericsson's Erlang/OTP Development Team. I do not want an anarchyfor computer language and operating systems. I prefer BSDism thanLinuxism in this sense; I think pieces of code should be rathercontrolled by the core people while sufficiently accepting improvementsfrom the other developers. The old Erlang/OTP daily snap archives, however, are no longersufficient to catch up with the daily development cycles. And manynon-Ericsson authors have put in their patches into Erlang, includingmine. So there had to be some systems to accept user feedbacks. Using an open repository system such as Github is a wise idea forincorporating new code into Erlang/OTP, and showing the official statusof modifications. Git is flexible enough to allow per-user andper-purpose branches. And Github allows forking between the users. TheEricsson's Team doesn't have to build and publicize its own coderepository system for Erlang/OTP, which will cost them significantamount of human and financial resources. And now I have an official requirement to learn Git; to catch up withthe Erlang/OTP development cycles. The definition of eventually secure systems 2009-09-02T00:00:00.003+09:00 I've been using Web services under a new assumption of integrity thesedays, which allows the data inconsistency during a span of a fewminutes. The designers of those systems allow such a relaxed conditionto data consistency, for putting higher priority to availability andtolerance to split database subsystems within a cluster representing anintegrated database. Then a question comes into my mind: what does it mean for a database tobe secured, while allowing unstable condition in a range of fewminutes? Of course guaranteeing unconditional access restriction is asolution to claim a database secure, provided each party who is allowedto get access to the database does not harm the integrity at all. Thissort of strict access limitation, however, is impractical for a publicsystem. So, a new notion of security, probably called eventuallysecure systems, should be introduced. But how? I still have noidea about this. Traditionally, databases are designed under the restriction ofAtomicity, Consistency, Isolation and Durability (ACID) for everyquery and update operation. The ACID policy demands locking of criticalsections between conflicting database requests and causes performancedegradation. On the other hand, Gilbert and Lynch [1]claim in their CAP Theorem for a distributed database, that thethree properties of a database will not be realized at the same timing:data consistency, availability, and tolerance to network partition.BASE [2], which stands for basically available, soft state, eventually consistent, is an example of anti-ACID designpolicies based on the CAP Theorem, giving higher priority toavailability and tolerance to network partition than the dataconsistency. Vogels [3] also explains the idea of eventual consistency, or aneventually consistent change of states, as an analogy to DomainName System (DNS), which allows the clients to query the distributeddatabase to see the inconsistency during the propagation of databaseupdate events, while the inconsistency will be resolved in a finiteperiod determined by the configuration of the replication networkbetween the database caches. While CAP Theorem, BASE, and the notion of eventually consistent systemsare effective to relax the boundary condition of data inconsistency formaking a very large-scale systems, those ideas will not solve the coreissue: how to keep the consistency of a cluster of a database in afinite predictable time range. I understand many applications do notrequire atomic consistency of data, especially those for casualconversation, such as Twitter or Facebook. I don't think, however, thata bank system can be created under the BASE principle, unless themaximum allowance of temporal data inconsistency or the maximum time ofeventual convergence are given and proven. And I think on running large-scale systems, things are often gettingeventually inconsistent and disintegrated, rather thaneventually consistent. I still wonder how we can solve this problemconsistently. References: [1] Gilbert, S. and Lynch, N. 2002. Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News 33, 2 (Jun. 2002), 51-59. DOI=http://doi.acm.org/10.1145/564585.564601 [2] Pritchett, D. 2008. BASE: An Acid Alternative. Queue 6, 3 (May. 2008), 48-55. DOI=http://doi.acm.org/10.1145/1394127.1394128 [3] Vogels, W. 2008. Eventually Consistent. Queue 6, 6 (Oct. 2008), 14-19. DOI=http://doi.acm.org/10.1145/1466443.1466448 Current and outdated references of secure C programming 2009-08-12T00:00:00.003+09:00 C is the modern assembly language for many architectures, and still themost useful computer language for me. C does not have a rigid grammarand has a lot of variants and local dialects, and have revised a fewtimes including the old UNIX C, ANSI C 1989 which first introducedprototypes, and C99. Finding out the de-facto standard elements of C is a complicated work.You can find a bunch of different indentation andwriting styles on C code. I do not recommend a specific coding style inthis article; I can only recommend you need to follow the mainstreamstyle when working in a project.Sometimes you have to read the books for discovering what is the most right thing to do.I recommend following books for C programming now: C: A reference Manual (5th edition) (2002), which has a lot of precise and detailed examples on ambiguous usage; Secure Programming Cookbook for C and C++ (2003), which explains what to do and what not to do to write secure code; and Advanced Programming in the UNIX Environment (2nd Edition) (2005), which explains usages of modern UNIX system calls and libraries. For practical programming, however, depending on books is not enough.Actually those books I recommended above are 5 to 7 years old as of2009, so if you want to know the cutting-edge details of programming,you should read the latest software. Consulting a C compiler manual andwell-written source code such as that of BSD kernels is a must if youwant to write an efficient code (those are freely available).One thing to which you've got to pay special attention is that booksare eventually but surely getting outdated. Books are not theWeb articles; they are static and will not change. The lifespan of areference book for computer science is typically very short these days,due to the rapid change of technologies. Books about C is not anexception either. And I should confess that a few days ago I decided to sell the followingold worn-out books because I found out them simply outdated (and Ino longer recommend the following two books any more): The C Programming Language (2nd edition) (1988) The Standard C Library (1991) The reason that I found them outdated were as follows: They are old, written in approx. 20 years ago, and they do notreflect the changes of C99 and other additional elements; Not mentioning secure programming at all, includingavoiding reference to non-existent data objects, preventing buffer overflows, limiting the length of a string; and The C library structure and source have been changed a lot for these 20 years. Frankly speaking, I loved those old books, especially which I referredto the most during my apprentice time of learning the language in thelate 1980s. Those books were the only source before the Web. I had torepeatedly read the old bestsellers many times to discover the details.I do respect the authors of those books. They are pioneers of UNIX and C programming.Nothing is eternal, however; and I suggest use to stop using outdatedreference books ASAP for every subject, not only for programming. Moving away from heavyweight blogs 2009-03-30T11:09:00.000+09:00 I admit I've been dormant on blogs. Instead I've been active on Twitter and Tumblr; you can be reactive and posting short messages on those so-called microblogging systems. Time is the most scarce resource for me, and I want something with smaller overhead to write. I will not entirely remove the contents of this blog, but I will not post a new article here, without further notice. FYI, the pointers for my microblogging URLs: (Note: the tumblr sites have been deleted by July 2009) jj1bdx: quote of technology Cyberperiscope Reviews (in Japanese) Twitter: jj1bdx (in Japanese) Twitter: kenji_rikitake (in English) Concurrently Chaotic (in English) Open-plan office and peer-monitoring socialism against creativity 2009-01-26T16:44:00.000+09:00 When I started working as a programmer after I graduated from college in 1990s, I was fortunate enough to have a wall-separated booth, thoughwithout a door. This is something which workers have taken for granted atresearch laboratories in the USA or Canada. But things have beendifferent in Japan, where I live and work.Having a separated space for individuals has been considered a luxury inJapanese companies, where people think space is money. So I should emphasize I was fortunate; because in Japan still corporateoffices are mostly open-planned: everybody seeing each other with nowall, whole bunch of noise, and is forced to listen to each other. I had to work in 1980s with an open-plan office in Japan as an inturn,and I thought working in the office would surely hurt my body anddegrade the quality of my thinking. If I were just moving around anddoing ordinary tasks, I wouldn't have considered it much. But I had tothink there for writing a technical report. So I thought something hadto be changed. I do not reject the idea of shared meeting space or the importance offace-to-face meetings. Those are vital factors of successful companies.But without a place for solitude, nobody would be ableto think. Without thinking, no innovation will come, and nonew idea will emerge. How can you think without being alone? Recently I've found an article on Web which says working in open-planoffice makes you sick and is hazardous to your health. A recent study of Dr. VineshOommen and his group in Queensland University of Technology showsthe following results: Results: Research evidence shows that employees face a multitude ofproblems such as the loss of privacy, loss of identity, low workproductivity, various health issues, overstimulation and low jobsatisfaction when working in an open plan work environment. Well said.Tom Demarco and Timothy Lister also write in one of theirclassics Peopleware (2nd Edition, 1999, Dorset HousePublishing) as follows (in Chapter 12): Management, at its best, should make sure there is enough space, enoughquiet, and enough ways to ensure privacy so that people can create theirown sensible workspace. I've read the 1st edition of Peopleware (published 1987) in 1989, so theworkplace privacy issue is well-known for at least 20 years.On the other hand, Japanese workplace has little changed for the past 20years. I still see many open-plan offices, especially amongnon-engineering workers. I suspect Japanese open-plan offices are designed for managers to putthe subordinates under surveillance during the working hours. This isan example of a dark side in Japanese workplace socialism. In a typical office layout, a manager in a team has the own desk besidesthe cluster of the desks for the team members. A team member can't takea rest or make a physical movement during working hours. I think thissort of desk layout does not respect the health of the team members, letalone the privacy or the productivity. I've found quite a few articles about this open-plan office sicknessissue on the Web. So I think this is a matter of concern for manypeople. Maybe this is a sort of backlash due to the recent economydepression. I'd rather work alone if I were put into an open-plan office every day again, solong as my brain and my ideas are the source of my income. The risks of systems left alone and untested 2009-01-10T08:20:00.000+09:00 Computer systems left alone unmaintained are a premier source of risks. Those systems may cause a serious crisis and a major service disruption. On September 14, 2008, All Nippon Airways (ANA), a major Japanese airline, caused the disruption of the ticketing service due to the cryptographic function software expiration (as they announced in the Japanese press release), which is logically assumable about the PKI certificate, according to the other Japanese-written press reports like ITmedia's and Nikkei ITpro's. The chilling fact revealed was that the ANA left the cryptographic function unused for 2 years and did not make a review about expiration at all when they activated the function for the terminals used by the ticketing agents. This is an awful example of software development indeed. I wrote about the service disruption for RISKS-DIGEST 25.34 just after the incident occurred. And recently I knew the article was quoted by another blog article yesterday. The expiration issue is not only about the PKI certificate; domain name registration is another source of expiration risks. An expired domain can be abused for phishing and overtaken by attackers. Software license is another good example. In general, Expiration is a part of overall misconfiguration. So when did you review the expiration date of the system resources under your control last time? Chain of distrust 2009-01-05T21:23:00.000+09:00 Communication is a collection of trust between the involving parties. Unfortunately, the trust is eroding in Internet, or in the society itself; and I see the emerging chains of distrust. An idea called Chain of trust is a practical implementation of authentication. Let me put it in this way; when Alice trusts Bob and Bob trusts Carol, then Alice assumes Carol is trustable. In this way, Alice doesn't have to directly authenticate Carol. Internet is another good example of chain of trust; each router assumes the peer routers will forward the packets originated from itself. But the chain of trust is not what should be taken as it is, in the real world. In the Alice-Bob-Carol case of the previous paragraph, the peer-to-peer trust relationship between Alice and Carol is not necessarily established; the existence of distrust between Alice and Carol is even possible, and they may don't want to talk to each other. Communication through a proxy is in fact quite common between the distrusting two parties. Should I call this a chain of trust? I should rather call this a chain of distrust. The current Internet is full of chains of distrust. Maybe I should rephrase it for accuracy; the chains of limited trust. For example, your employer will not unconditionally trust you to protect the employer's privacy, so you have to communicate outside the employer's network through a firewall, usually made of packet filters and proxy servers. Your employer gives you a limited trust for the external communication. This sort of limitation may cause your distrust to your employer, but the employer usually considers this is a security feature to protect the relationship with you. The difference of interpretation to the situation of limited trust can be a source of distrust. In a set of trusted parties with a limited size, each party does not have to spend time on authenticating each other for every packet they communicate with each other. The trust is proven through the physical connection and perimeters. Internet's packet forwarding system extends this idea of physical connection to the chains of trust by reliable communication with discrete packet deliveries, and the idea has worked well in a limited community where the people are trustable with each other. The end-to-end principle [1] has worked so effectively that the engineers of Internet firmly believe in it. The reality we are facing, however, is that the people are no longer trustable with each other and rather distrusting one another. People are seeking for a safe haven by creating a chain of distrust, which is apparently a false sense of security, considering that the chain of distrust is easily broken if the proxy between the distrusting two has a malicious intent. We are heading into the very difficult times, where the security engineers ought to secure the chains of distrust as well as the chains of trust. Reference: [1] Blumenthal, M. S. and Clark, D. D. 2001. Rethinking the design of the Internet: the end-to-end arguments vs. the brave new world. ACM Trans. Internet Technol. 1, 1 (Aug. 2001), 70-109. DOI=http://doi.acm.org/10.1145/383034.383037 Working CW stations at the end of the year 2008-12-31T16:59:00.001+09:00 I've set up an amateur radio station again this week, and worked some stations, including some friends. The transmission feeder is a temporary one, so I will disassemble and reassemble the system soon, after the new year vacation time is over. Amateur radio for me since 2002 is mostly CW (Morse Code). CW is the most primitive but still practical digital communication form. The bit rate is about 10 to 25 bps (no kilo, mega, or giga) (that is 12 to 30WPM) and it's very slow indeed. You still can send something meaningful over CW, and it has been used for more than 100 years. See how people are using Twitter these days. CW has already been phased out from the mainstream of professional communication systems, but it's still a viable backup, and ham radio operators still use it over shortwaves to exploit the maximum possibility of communication, to overcome the difficulties of natural and artificial noises, and significant path loss. On the other hand, the latency is minimal; only the path distance between the peers determines the delay time. It's the oldest chat system in the world. The hardest part of CW is that you need to learn and have a lot of practice to listen to it. It's like learning another foreign language. Some eccentric people including me, however, choose this road less traveled, for many reasons. Many old operators may find out their younger days of enthusiasm towards radio itself. And many hardware hackers will use CW for realizing the simplest but useful radio system with a pair of home-made transmitter and receiver. Whatever the reason it might be, still quite a lot of people devote themselves to CW. And I am one of them. Working on CW is simply a good time for me. Learning for a faster CW listening stimulates my mind a lot. And CW reminds me of the very basic issues of digital communication; reliability, bandwidth, stability, latency, and practicality. The procedure of sending and receiving letters by yourself gives you more time to think. I'm a computer engineer and scientist, and technologically CW is an archaic thing. The principle embedded, however, is still effective to be a fundamental motivation of pursuing engineering and scientific skills; to explore the road less traveled and discover the new world. The following article of Jeff, KE9V may help you to understand why some old amateur radio operators are still enthusiastic to operate on CW: The Road Less Traveled 73 and have a happy new year 2009. Kenji Rikitake, JJ1BDX(/3), JO3FUO, N6BDX and JQ2KST My Influences 2008-12-06T10:23:00.000+09:00 My influences on programming, Internet, and computer skills: Tsuneji Rikitake, my father: who taught me basic math, FORTRAN, and financed my startup (of my professional career, indeed) Paul Vixie and DEC NSL people: for the primary idea of firewalls and Internet systems administration, and the proper engineering attitudes toward problem solving Daniel J. Bernstein: on re-thinking Internet protocols from the very beginning, especially on DNS Bruce Schneier: for the basic philosophy and principles of security, not only for computer systems UNIX gurus on Bell Labs and BSD communities: for the programming suite and styles, including C, awk, and Bourne shell Joe Armstrong and the Erlang programming communities: for letting me know a practical message-passing-based concurrency I know there are many other people I should put on the list, but I will just keep this list small. My salutation to Chad Fowler and Kevin Smith for reminding me of this topic. A quiet life without a TV tuner 2008-11-16T17:28:00.000+09:00 I've been spending time without TV at home since this (2008) September. And I recommend you to do so too for living a quiet life. I decided to stop watching TV regularly; no airwave, no satellite, no cable channel at all. Actually the condo building I live is cable-TV ready. But I decided to quit. I gave our LCD TV to my wife's Mom. We instead listen to the radio when we're at home and having a dinner or lunch and we want some news. Most of airwave TV programs are junk. And most of non-airwave (that is, paid) TV programs are also junk. And we know most of airwave radio programs are junk. But some of them, including news and classic music programs of our national broadcaster (in Japan), are fortunately not. I carry around a pocket radio when I travel in Japan to listen to the same night program called Radio Shin-ya-bin, a rather-quiet-and-calm program without ads. We still want to see the DVD contents. We have a DVD player. So I decide to newly buy and install a new PC display instead. It accepts DVI, VGA, component and NTSC video, and even HDMI. It has 1920x1200 pixel resolution; an excessive spec for DVD or an old PC. It doesn't have a TV tuner, however. It doesn't have speakers either, but that is OK, because I installed a pair of speakers and an amplifier with AM/FM radio tuner installed. So no problem for enjoying audio. The good thing about not watching TV while on a family meal is that we rather talk and make a lot of conversations. And we started to read a lot of books and talk about them. It's like living in the world of 1950s or 1960s, though we still use PCs. We have also quit subscribing newspapers since this April. I decided so because even Nikkei Shimbun, let alone other papers, were carrying junk articles these days. The New York Times and The Economist have already allowed people to read their articles online freely, as other media companies follow. So I told my wife we didn't need a newspaper subscription at home anyway. She first complained, but eventually she also learned to enjoy reading something instead, or to spend the time for being exposed to mass media to something else. We don't have any game machines such as Sony's PlayStation or Microsoft's Xbox. Our eyes can no longer follow or respond to artificial graphic pictures. We mostly use PC to read letters and design presentations; we don't make active 3D graphics and we will not unless we need to visualize something for our works. And of course we see visual presentations on YouTube and elsewhere. The good news is, however, they are on-demand only. Unless you explicitly tell PCs to show them, you don't have to watch at those on-demand videos. Why choosing Erlang for concurrent processing 2008-11-13T15:55:00.000+09:00 A few weeks ago I was reading a blog article called Erlang, the next Java. While I agreed with the author's views, I felt something was bugging me in the article, which eventually lead me to write this blog article. I didn't choose Erlang just because it's a functional language. I have chosen Erlang to learn because Erlang programming requires and has to enforce the programmers to write the concurrent code. Concurrent programing should minimize the number and size of critical regions and bottlenecks. And I think writing such programs is very difficult without the assistance of the programming language. Many programming languages claim or plan to be capable of running a part of code based on concurrent processing. For example, Ruby will incorporate distributed storage mechanism called Roma and the task administration subsystem called Fairy. Another good example is Haskell concurrency: it is implemented as a language feature called Parallel Haskell. I think, however, introducing concurrency while allowing programmers to write code to use shared memory will cause a lot of problems. Joe Armstrong has already described his concern on shared memory in his blog. I support his arguments; those arguments an important part of the reason why I decided to learn Erlang. I should add another problem programmers will face when dealing with code allowing shared memory; rewriting the code for removing shared access to run it efficiently on concurrent environment will be an incredibly difficult task. Unfortunately, most existing languages have already had a lot of code written assuming shared memory areas. For example, C code with extern variables implicitly assume those are shared between the functions in all source code files linked together. I assume I can hardly find any set of C code without using an extern declaration. I have learned that even Common Lisp has special declaration for the variables, which allows multiple functions share the same object, out of lexical scopes. Another example of shared-memory concurrency is the operating system (OS) threads. C/C++/Java threads inherently share the parent OS process address space and environment in common. Python has the Thread Objects. While OS threads often ease implementation of concurrent servers by reducing the task switching time, the semantics is implicit and error-prone. I understand and agree that sharing objects itself cannot be completely eliminated under the read-world constraints of processing and communication timings between programming language functions. I think, however, that programming languages should help the programmers to minimize writing code including shared memory areas, which will turn themselves into critical regions. Erlang imposes necessary restriction on avoiding implicit data sharing between functions by: prohibiting multiple variable assignments in a function; enforcing and helping the programmers to conduct message-passing programming between functions, by not providing any implicit data-sharing facility between the functions; providing fast task-switching capabilities, by giving the definition that functions are the minimum concurrent execution units; and restricting the usage of data-sharing facilities between functions to the minimum, such as process dictionaries, ETS, DETS, Mnesia, and the global naming service shared between connected Erlang nodes, by requring explicitly writing so in the code. In short, I think adding concurrent features is not enough for concurrent programming; prohibiting non-concurrent habits and enforcing writing concurrent programs are necessary as part of the programming language specifications. I believe concurrent programming under coming multi-processor environment is only possible under such a hard-liner attitude to the programmers. I feel programmers including myself are very conservative and rigid to change their sequential-programming habits. And actually, I was one of those programmers who did not recognize the urgency of learning concurrent programming in 2007. Programming language design and security 2008-08-31T11:07:00.000+09:00 I suspect quite a few programming language designers just don't care about the security of the languages and the systems built by them. Maybe I'm too pessimistic, but a recent experience brought me the idea. I was an audience at an event called Lightweight Language Future yesterday in Nakano, Tokyo, Japan. I only attended the first two sessions. The second session was a panel discussion about the programming language 100 years later. While the discussion was technically intriguing, I heard not a word about security at all during the panel. Two of the panelists (names withheld because I don't want this article to be a personal attack to them) were the principal designers of popular lightweight languages, so I asked them in the Q&A session about how they would deal with the security issues which would become more serious in the future. One of the answers surprised me; the person told me that VMware would do. I understand virtualization in various operating system layers, including VMware and FreeBSD jail, is surely a good hack to control the damage area in the whole system to run a public access system. I think it's not sufficient at all, however, to prevent future security incidents; if a programming language is potentially vulnerable to attacks, the systems built by the language will be also equally vulnerable. I think programming language designers are not exempted from the responsibility of (at least trying to) making computer systems secure. If I could hear more constructive ways of solving the security problems in the answer from one of the panelists, such as: protecting the language from the buffer-overflow bugs; preventing the garbage collector from crashing even in a hostile environment where the external attackers try to deliberately manipulate the pointers or variables to hack into the protected memory area; implementing a syntax or semantics validation system in the programming language so that the programmers can apply it to validate the external data, such as those from the network-connected users; or preventing the race condition, deadlocks, or any sort of resource starvation by the intentional or unintentional programming; then I would have been much more convinced. But now I should suspect that quite a few programming language designers just don't care about the security consequences of the features they build into the language. I don't have a definitive answer to solve the issues of unsecure systems built from unsecure programming languages. All I can write now is that a security-aware programming language will surely help making a secure system. Note that security is not just about privacy and secrecy; it's also about the reliability and fault-tolerance. Note on 4-SEP-2008: one of my friends later reviewed the panel recording and told me that it was not VMware but VM layer that the panelist answered to me. I still think solely relying on the VM layer is insufficient for building a security structure in a programming language, though. About myself: 1965-2000 2008-08-19T12:57:00.000+09:00 This self introduction is a reprint of an article once posted to my own website http://www.k2r.org/kenji/ as a part of my self introduction. I was born in 1965. I grew up in a Japanese urban community in Setagaya-ku, Tokyo. The whole things changed in 1974 when my father decided to go to Boulder, CO, USA for a visiting professor job at Cooperative Institute for Research in Environmental Sciences of University of Colorado at Boulder. For the first 6 months of my American life, I didn't speak English at all. We lived there for 15 months, so I was a lousy kid speaking broken English just before when we returned to Japan. I still think the life in the USA was the best of my times. I've never lived abroad since then. I've been keeping up my English proficiency by listening to and watching English-speaking broadcasts, and reading English Web articles. After coming back to Tokyo, I found out myself I was fatally incompatible with and just way too outspoken than other kids around.The totalitarian and socialist society and elementary schools of Japan did not and still do not accept me. They told and still tell me that I was and I am so-called contaminated with individualism and American way of thoughts. So I had to make a difference to survive. I didn't have much advantage from the other kids, except for hacking around the radio and audio devices, writing some code on the papers (no desktop PC existed in 1975 in Japan), playing around with electronic circuits, and having no fear for speaking English. I was just a geek. My father Tsuneji bought me a FORTRAN book and a shortwave radio in 1973, the slide rules and electronic calculators in 1974, and a bunch of math books. I learned them mostly by myself. I went to Musashi high school, a private school well-known for the liberalism. Soon I became a programmer of Apple II in 1979. My programming experience was mostly on 6502 assembly language, 6K Integer Basic, and the GAME programming language. ASCII Magazine bought my GAME-APPLE compiler in 1980 and I earned about JPY200,000 by the sales.During that time I met a lot of talented hackers. Many of them are active in the Japanese Internet R&D scene. In 1982, my right eye was badly damaged with the retina detachment so I had to stay away from computers. I started to make techno music instead of computer programs, using later well-known techno gears such as TB-303 and TR-909, until 1986. I loved making music, but I didn't feel I belonged to the music industry. I think I made the right decision,after discovering the whole mess of the ripping-off business models. I went to University of Tokyo in 1984 and there I experienced the harshest part of Japanese career competition. I managed to be accepted by the faculty of Engineering to learn instrumentation physics. I later studied information engineering for my Master's degree. Life in the university was no fun; hanging around with real hackers and real systems were much better. In 1985 after NTT finally opened up the phone lines to public, I discovered international computer networks and online systems. I soon became an e-mail and BBS junkie. In 1987, I started working with Jeffrey Shapard, who was running an English-speaking system called TWICS, for the VAX/VMS system administration, until 1989. During the TWICS days I met Joichi "Joi" Ito, an English-speaking multi-talented Japanese activist. Many hackers went for UNIX, BSD, VAX and Sun in 1980s. I was also one of the root-privileged users of a packet-radio UNIX systems, learning vi and hacking with the code. Having an account on an UNIX system was a cool thing. What made my difference was that I ported the UUPC/Extended software package for NEC's PC-9801 computers. It was a version of UUCP software,which performed automatic file exchange and email address relay between computers. And I was a licensed radio amateur (as JJ1BDX since 1976), so I made some gateway software packages linking ham radio TCP/IP mail/NetNews systems and the UUCP-based systems. In 1990, I joined Digital Equipment Corporation (DEC) Japan, to write the core software of VAX/VMS. During that period, I had a chance to exchange messages to many Internet gurus, including Paul Vixie, a DNS guru and Internet leading technologist, who was working for DEC then. It was really nice to learn the philosophy and principles of Internet from those hard-core hackers. During my 2 years of Digital career, I was struggling with the corporate bureaucrats, and too many overworking and commuting hours which were eventually killing me. I met Kyoko in 1991, and we decided to get together and married. She told me she had a new job in Osaka from April 1992, so I decided to move to Osaka too. I was looking for an Internet job there, and Kazuo Hirono hired me for one of his research team at TDI Co., Ltd., a Japanese software service firm. We started to build a research lab in Kyoto and the lab was operational from 1993 to 2000, also as a part of WIDE Project Kyoto NOC (POP). Hirono-san taught me a lot of things, and he was very tolerate on my rather individualistic and outspoken communication style. (Hirono-san passed away on June 2007. RIP.) (Originally written in March 2008) nsplit available as a Google Code project 2008-08-15T14:59:00.000+09:00 My Erlang code nsplit is now available as a part of Google Code projects. Link on this blog about the code is here. Outsourcing my DNS lookup service to OpenDNS 2008-07-24T16:32:00.000+09:00 DNS subsystem is a real headache to manage. You will realize that once you have to tweak the system, especially if you want to incorporate your own filtering/access-blocking rules. And the DNS Cache Poisoning. It's a real threat. While I believe my DNS subsystem is quite safe under the good-old djbdns servers, I am now testing the OpenDNS, a DNS cache service provider. One of the good things about OpenDNS is that they even allow a single-IPv4 address network to be individually managed, even it's dynamically allocated, as in most of the cases for non-static-IPv4 users. If you can trust your ISP for the DNS management, you are on your own. But if you can't or don't, OpenDNS is a good alternative. I notice many ISPs still have not changed their DNS cache servers to prevent the poisoning attack as of today (July 24, 2008); using OpenDNS from such a mobile networking environment will make the whole DNS access much secure. My allegation of spamdexing is finally dropped 2008-07-16T14:04:00.000+09:00 It seems that the Blogger admin finally dropped the allegation of spamdexing from http://macrofield.blogspot.com/. It took more than 2 months, but less than 3 months, to solve this issue. Of course, I removed the suspicious META refresh tag from the alleged page. Links: The report of false detection of spamming Meta refreshing considered spamdexing Erlang: parallelism without modifying code 2008-06-10T10:14:00.000+09:00 The Erlang programming language has been getting popular among programmers, for the capability of handling massive number of connections, such as the chat server used in Facebook. One of the basic ideas of Erlang are that minimizing the side effect of assignments by prohibiting multiple assignments into the same variable. At first this looks a rigid restriction and memory-consuming practice, but once you learn the native list structure of Erlang (mostly the same as in the Lisp language) and the tail-recursion control flow, writing the Erlang code is not too difficult, while retaining the conciseness and the performance. I've written a code of parallel mapping of a function to a list. The word mapping means applying the same function to each member of the source list and obtaining the results which retains the same sequence as in the source list. If the function does not have side effects (such as changing the values of shared data structures), the operation can be parallelized by splitting the list into the smaller sublists and invoking the mapping process for each sublist. Erlang has the supporting libraries of invoking a process in a distributed Erlang node running in multiple computers, so the parallel mapping function is the simplest but very powerful tool to experience the collective computing power of parallelism. The code is available here as a tar archive, of one file of Erlang source code. Installing FreeBSD 6.3 and 7.0 to Panasonic CF-R4 2008-05-17T15:33:00.000+09:00 I had difficulties installing FreeBSD 6.3-RELEASE and 7.0-RELEASE to Panasonic Note PC CF-R4. This machine has been known for having an incompatible BIOS which prevents FreeBSD BTX 1.01 or earlier to boot. Recent FreeBSD 8.0-CURRENT snapshot ISO images are with BTX 1.02, which allows CF-R4 booting from USB CD-ROM drives. So I tested the following procedure for the 6.3 and 7.0 releases and the both releases were successfully installed: boot from a CD-R(W) with 8.0-CURRENT-200805-i386-bootonly.iso (or later) exit the boot prompt screen to the loader prompt (by pressing 6) before the 8.0 kernel boots itself enter unload to clear the loaded (8.0) kernel replace the CD-R(W) in the drive to the target installation image (e.g., Disc 1 of 6.3/7.0-RELEASE) enter boot-conf to load the target kernel and install I think this is the compatibility problem of BTX. I also think this is so far the easiest way to boot from the USB CD-ROM drive for CF-R4, because generating ISO images for previous releases with the compatible BTX loader will be a big hassle. You may give it a try if you want to boot from a USB flash memory. Learning Erlang 2008-04-26T15:21:00.000+09:00 I've been learning the Erlang programming language. I've tested an IPv6 address-to-domain-name conversion for the reverse lookups. I made a presentation at the 1000speakers:4 conference on April 26, 2008. The presentation slide set in PDF is here. The days of Neuro Net Recordings 2008-04-09T20:39:00.000+09:00 Neuro Net Recordings (NNR), an online techno-music distribution project, is one of the most important music projects of my life. Many music files of NNR have been available at archive.org as a part of the open source music archive. It was founded by Tomotoshi "Ackie" Akiyama and other core members including myself, in 1994. I was the Internet Archivist, or the one who handled the online distribution channels over Internet. NNR had the free online distribution policy from the beginning: NNR had the non-exclusive distibution rights of the music files in the various available formats on the Internet. It was quite similar to the Creative Commons BY-NC-ND license, which means preserving the author's credit, non-profit use only, and changes not allowed during redistribution. NNR was not really a for-profit label, but most of the albums of NNR Active Potential Series were distributed by cassette tapes with reasonable price (JPY500 for Active Potential Number 0). NNR also released one CD album called Neural Networks Vol. 1 in 1996. So NNR was really an independent label, though NNR consistently distributed the music files towards the potential listeners on the Internet. NNR once had a redistribution agreement with FranceMP3.com. Jean-François Fecteau has been a long-time supporter of NNR activities, and he has played NNR tunes many times in his program Le Vestibule from CFOU-FM in Trois-Rivières, Québec, Canada. The distribution format NNR chosen were RealAudio, MPEG-1, Ogg Vorbis, and MP3 finally, because MP3 technology license owners allowed exemption of the MP3 technology royalty fee to the business with USD100000 or less annual revenue. Ackie kindly sent me the past DATs for mastering NNR cassette tapes, so I rebuilt and remastered the CD-R master discs from the DATs. I learned the remastering skills using Syntrillium's Cool Edit (now Adobe Audition) and other tools. I also performed real-time streaming events called Neuro Net Radio twice on 1998 and 1999, which were actually DJing from my house over RealAudio streaming, playing only NNR tunes to avoid dealing copyright issues with the copyright holders. Many professional music creators contributed to the NNR music archive before and during they became really professional. Some of the artists are: NISH, Leopaldon, Takuya Kitani, and Warping Allergen. I find many NNR contributors are still actively seeking their other opportunities in their lives. In the age of iTunes and Amazon.com MP3, no professional musician can survive without distributing their music online. Streaming music radio stations like Soma FM are doing competitive business. Many people in the so-called music industry, however, still do not accept online media, and I feel quite sad about it. I've already been fed up with the stagnated copyright issues in Japanese music scene either. So I decided to quit distributing NNR files on my own in 2004. Fortunately, archive.org generously provides the storage space and other goodies to the free-music distributors, so I decided to put NNR and my music pieces there in 2007. I know my talent on music is quite limited and is not competitive enough to survive as a professional musician. But I also believe that does not limit my rights to freely distribute my own music. I've learned a lot through my NNR activities and I appreciate all those who helped NNR. Fumi Yamazaki suggested me to write this, as a case study of Creative Commons activities in Japan. Thanks, Fumi. META refreshing considered spamdexing 2008-04-04T09:16:00.000+09:00 I am writing this as a memo for myself, and a message to Blogger.com reviewers of my old (blocked) blog at macrofield.blogspot.com. I realize the whole issue is about Meta refreshing of HTML. This has been a common practice for a long time to redirect users from an old page to a new page. The problem of this practice, however, is that users will have difficulties on intervening the redirection. I embedded a Meta-refreshing tag in my old (now blocked) blog for an automatic redirection. As seen in the Blogger.com's definition of spam blog, this redirection is considered spamdexing, a violation of Blogger.com service terms. I think no one likes spamming any more and neither do I, but I should confess that I didn't study well on how Meta-refreshing is considered abusive these days in the HTML writing practice. I will remove the embedded META tag as soon as Blogger.com allows me to reedit the contents, or to be specific, to reedit the layout HTML tags of the old blog of mine. And I suggest you all to stop using Meta refreshing. Being misunderstood as a spamdexing is doing nothing good for you, whatever your intention might be. Possible false detection of spamming on Blogger.com 2008-04-04T08:05:00.000+09:00 This is an administrative announcement. Due to the automatic classification of possible blog abuse on Blogger.com, the old URL of this blog, macrofield.blogspot.com. has been disabled by Blogger.com. I've already requested Blogger.com to unlock the contents under the old URL. I will make a necessary fix ASAP. I'm sorry for the inconvenience. Note: on July 2008, I have confirmed this blockage has been dropped. Gauche: yet another Scheme-a-tic diagram of programming (self-)education 2008-03-23T10:32:00.000+09:00 I feel programmers in the 21th century prefer so-called dynamic, scripting or light-weight programming languages to the good-old static, compilation-based, and heavy-weight ones. In this sense, I belong to the old-style programmers, who like C, awk, and simple Bourne Shell. So far the most complex light-weight language I've used is Perl. I like definitive or pre-defined syntax and keywords than those which may change the meanings by the programmer's definition, because the dynamic definition will make the program too complicated. Maybe my brain is losing flexibility, but I'd rather want to read something which can be easily decoded than the cryptic ones. I didn't have a chance to formally learn programming, however, during my education. This is both good and bad: the good thing is that you don't have to spend time on learning subjects which you won't use throughout the rest of your life; and the bad thing is that you may have no chance to know the coding practice much efficient than what you usually write. Some programming languages still irritate me. I should confess I don't like Prolog. I will be a very poor programmer if I have to write only rules between the objects than the way of moving around and cutting and pasting them. Thinking about the backtracking facility is a nightmare for me. I'm 99.9% (if not 100%) filled with the idea that every statement of a programming language will have a side effect, or may cause changes of the contents of the preallocated memory space. I still don't believe in the idea that no assignment is needed for a real-world computing. I still think, however, the idea of functional languages which is focused on the relationship of data than copying and manipulating them back and forth, is an essential part of the modern computer programming. Abstraction work of data often simplifies the complex problems. Writing things in recursion than an arbitrary loop will simplify the code and often makes it efficient. Recently I've learned many young programmers are learning a lot from LISP and Scheme, and I've heard some of them have learned Scheme in their formal education process. I welcome that trend and I hope it continues to grow, because in the real-world programming paradigm there's large room for optimization by introducing Scheme-style abstraction, such as describing an XML document in an S expression. Gauche, an implementation of Scheme by Shiro Kawai and other developers, is a well-balanced mixture of the LISP-Scheme tradition of simplicity and a Perl-like feature, such as being able to handle Unicode, regular expressions, network sockets, and simple byte-aligned buffer structures which is essential for network programming. Gauche is a viable alternative to scripting languages like Perl; and it's still a Scheme implementation, so an old-minded programmer like me has many things to learn and enjoy. Gauche has a lot of modules or external libraries to handle real-world problems, such as manipulating database engines and HTML/XML data structures. It took me only three hours to rewrite a Japanese input method server program in C by Gauche, thanks to the modules. Gauche and Scheme designers also know that programmers want to use assignments, so an old-headed guy like me didn't have to scratch the head forever. In 1989, when I first met a Symbolics LISP machine, it didn't attract me much because the machine was something completely different from the UNIX workstations, which I wanted to use to communicate with my friends and gurus. I realize, however, things have changed a lot now; you can play around and even solve the real-world issues on a LISP-Scheme processing environment like Gauche. The vast increase of processing power including the CPU speed and the amount of main memory make this possible for everyone. And I think it's a new Scheme-a-tic diagram for a competitive programmer; you will become much more productive after learning practical data abstraction. Gotchas, gotchas, gotchas 2008-03-03T19:29:00.000+09:00 I've learned a lot of hard cold facts on how to deal with multimedia systems. Here are the examples: White fungi can grow on a CD medium surface and even often render the media unreadable, though in most cases the medium can be rescued by wiping out the fungi. (Be careful not to damage the acrylic surface!) Windows XP goes into an infinite loop (100% usage of a CPU core) to try to read an unreadable CD medium. If you don't have a multi-core machine, you'll be getting into a trouble. Why the CD/DVD drive cannot tell you that the media seems to have a problem, rather than just simply retry and wait for the timeout periods? iPod 5G firmware Version 1.2.3 can crash and reboot forever, just because the album title field of an MP3 file contains an unreadable/irregular character. The iTunes 7.6.1 for Windows is running smoothly without any error message even with the MP3 file, so you can't tell what's really wrong until you try-and-fail the whole possible cases. You cannot easily attach still pictures to an audio file without external software. And I realize that's the meaning of the word authoring of a streaming video. You have to make a set of still pictures into motion pictures or a video stream first first, before syncing with an audio stream and make them a video stream, to be broadcast on the Internet, such as on YouTube. (A good news is that you can do this by a set of the open source software, such as mjpegtools and Avidemux.) Gracenote CDDB has been broken for WinAmp, so I have to migrate into iTunes, just to pick up the metadata of each song I'm going to put into my lovely iPod. I like my iPod very much, but iTunes for Windows is really a big slow program and I hope it was a bit smaller and faster. And I even learned some security and systems administration tips: Many of PCI Serial-ATA cards cannot be used for operating systems other than Windows. A 13-year old hard disk can spin up and is readable even if you once put it in your closet and left it unplugged for 10 years. As of 2008, you can hardly find a SCSI interface card for a PC, at least in Osaka. Then I remember the truth: your computer could be the number one device to consume your time and kill your productivity. So how many gotchas do you get every day? Ten? Or a hundred? And how many hours or even days do you lose? Teleworking technology: we already have it 2008-02-21T12:25:00.000+09:00 I've been thinking about months for what technology can help people to discover and get the most out of teleworking. I'm getting closer and closer to the following conclusion: We've already built teleworking infrastructures, at least in my home country Japan, and many other nations and regions which have already deployed the Internet. Period. We need to focus on how to break the barriers and remove the impediments to people to embrace teleworking technologies as their own tools. Let me simply put in this way: teleworking is no longer something special. For example, writing this blog itself is a very good example of teleworking technologies. All I need is a decent Web browser capable to run JavaScript for Blogger's writing environment; and I can publish and show you what I write, like what you read now. Collaboration is getting more and more easier also. LUNARR allows you to flip back an electronic document and let you and other people write something on the back. This is a rather intuitive way to co-author a document, which I had never thought about until I saw it. Geeks will do the whole things in more geeky ways using SSH, rsync, CVS, subversion, or whatever else for software development, but it's not only for geeks anymore. Anyone who has to write a document can do it on a teleworking environment. So let's stop saying "I'm not allowed to work from (wherever you want other than your office) because blah blah blah..." and talk how you can leverage your productivity from introducing teleworking into your lifestyle. If you and your employer think teleworking is something restricted for special people, that mindset is archaic, and no longer applicable to the 21st-century version of modern lifestyle. And let's not take communiting as a duty; it should be a choice and it has to be, to reduce the gross amount of time and energy wasted by being forced to commute. But beware that teleworking is not necessarily a duty either; don't do it like this as in an xkcd article. Don't leave your partner alone just because "something is wrong on/with the Internet" (grin). The art and limit of dependencies 2008-02-17T15:07:00.000+09:00 If you want to get a job done, you need to list up the necessary things and tools, and procedures to use and apply them. In other words, you have many dependencies on them. In a modern computer software development, no piece of software can withstand without any dependency to other pieces. You do not want to make a C program without the standard I/O library. Dependencies to the tools are also important and critical; if your codes include those written in FORTRAN, C, C++, and Java, you need the four compilers and language execution environments. Computer programmers have been making the tools to automatically resolve the dependencies. make is a popular one, derived from the UNIX programming environment, which parses the rulesets called Makefile and determine whether if you need to rebuild a result from the source files, by comparing the timestamps. If make finds out one of the source files are newer than the result, it will invoke the command to rebuild the result. Version control systems such as subversion, CVS, and RCS, are another good examples of dependency management tools. You can save the history of changes on a file, a directory, or a set of directories. You can make a software package by checking out a set of tagged files; you can even make multiple branches of a code. FreeBSD operating system has its own dependency management system of externally-contributed programs called ports and packages. A port means a set of rules, configuration files, and source codes necessary to build a program. A package means the derivative of the port, built by another computer. Many essential parts of FreeBSD subsystems, including the X Window and Perl programming language, are installed as packages, because they are not considered as the core parts of FreeBSD. I'm always pleasantly surprised when a very complicated port, such as Japanese version of LaTeX, a typesetting and documentation set of programs, can be built without major glitches, including the automatic installation of depending programs, such as Ghostscript. For most of the ports, FreeBSD volunteers are always doing the outstanding jobs. I feel very much annoyed, however, when I have to untangle the web of dependencies when the installation/building of a software toolset from the scratch. Unfortunately, Xorg 7.3, a free implementation set of X Window programs, fonts, and tools, was not able to build in my environment. I had to copy three prebuilt sets of files: include files of C/C++, shared/static libraries, and the very basic fonts, to finish building the necessary programs. In my case of Xorg kitbuilding, some very old fonts existing since late 1980s were not successfully compiled and converted into various ISO8859 part codesets. And I could not build the font handling libraries. This glitch killed the whole automatic compilation task, which was supposed to untangle the enormous list of dependencies. I also found that the target directory name was merged to /usr/local from the traditional /usr/X11R6, and I wasn't sure whether if I moved the old files and subdirectories under the old directory safely to the new directory. Untangling the web of dependency is a very hard task, since the parameters you need to examine are scattering around all the directories, in all the ports, and you often also have to set an environment variable to do something extraordinary. While I can guess what I should do because I've done a lot of this kind of tasks, I never want to do this at any time because scanning your memory and poking around the files are very much mentally painful. In a recent hostile computer environment, vulnerabilities can easily sneak into such a complicated and unformalized tasks. How do you manage your computers? Do you care about the dependencies of the tools, programs, configuration files, and other objects? Accountability of an SNS user 2008-02-10T16:20:00.000+09:00 In a major Japanese SNS mixi, I feel I have been always exposed to a strong peer pressure, which I find common in Japanese society, that you should not argue with others unless you really want to break up with those people. This sort of fuzzy feelings covers over the whole society of Japan, including that on the Internet. I really don't like the fuzzy Japanese social atmosphere, though I don't favor the hostile and negative environment which I always find in anonymous and open bulletin boards and blogs in Japan nowadays either. I always want to have a creative discussion with a constructive criticism. People tend to go the opposite way, however, in most of the times. This is what I've learned both from the real and Internet communications. One of my friends claimed to me that the warm fuzziness surrounded him in 1980s in the Japanese online community of bulletin boards was something completely different from that after the popularization of Internet in Japan. He told me the fuzziness which I don't like actually helped the self-governance of the users. I asked him why. He told me that the bulletin board operators could individually locate and even persecute each account holder, by exchanging the written agreement in paper, and charging access fee individually from each user's bank account, automatically drawn each month. He said he had many anonymous or pseudonymous friends there, because he could trust those people even without knowing the real names, by the guarantee of financial and social accountabilities for each users that the BBS operator provided. In other words, the participants were supposed to behave nicely online, and the participants could prevent each other from the extreme cases of disputes. The friend and I agreed, however, that you could no longer expect such a high degree of individual responsibility on the current SNSes and other Internet communities. Many people belong to many different systems, and those systems are usually supposed to keep the personal information undisclosed except for a legal request for the authorities. This means all disputes between Internet participants have to become legal issues, or lawsuits. And even if you win a legal battle, you can't really collect the compensation justified by the law, unless the law enforcement officers really pursue the defendant to do so. The legally-responsible individual of one of the largest anonymous BBSes in Japan keeps refusing paying the money he has been requested from the court, though he loses on multiple lawsuits, and he is actually winning the battle by not adhering to the legal requests, because those lawsuits are all civil law issues and not the criminal ones. This is a good example of the limitation of depending resolution of individual disputes over Internet on the legal procedures. Forcing people to use "real" names will not solve this problem at all. How can an SNS user be held accountable, for his/her legal and social activities? This is not a direct technology issue, but technology can and must help the issue to be resolved. And I think tagging individuals and restricting online identifiers do not necessarily effectively work on this issue. CALLing disaster during MySQL upgrade 2008-02-08T08:20:00.000+09:00 I've been upgrading one of my servers for the daily use, by migrating the running environment between two PCs. It's not a mission critical server because I don't run a public service there, but I need to do the upgrade carefully anyway because the version number of running FreeBSD and other applications have been changed. I want to keep the old environment as long as possible to use it as a reference, so I'm doing the migration manually by recompiling and reconfiguring the software. One of the glitches I faced was about MySQL. It's not about the bugs in MySQL, because the database server load is very small. I was trying to transfer a dataset from the old version 4.0 to a new version 5.1 software of MySQL. The mysqldump result of the old 4.0 output didn't get through and reloaded into the 5.1 server. I could not even perform CREATE TABLE. The reason: the column and database identifiers were not properly backquoted. The database was for my amateur radio activity. Amateur radio stations have callsigns, and in a popular contacting log exchange format called ADIF, the other party's callsign is represented by the identifier CALL. Unfortunately, MySQL 5.1 made the word CALL a reserved word, while in the 4.0 version the word was apparently not. This is a tragedy for an amateur radio enthusiast, and a careless programmer like me who tends to omit proper (back-)quotation. After an investigation for a few minutes, an idea of referring to the result SHOW CREATE TABLE from the 4.0 server came into my mind. I did it and fortunately the table definition was properly backquoted, so at least I could rebuild the database skeleton. The dumped data was a set of INSERT INTO statements with the VALUES and they were all properly quoted, so I could rebuild the database. The new mysqldump command's output of MySQL 5.1 looks much better and properly backquoted and quoted all the necessary strings, and even put the SQL statements to lock and unlock the database. The entire dataset is represented by a single INSERT INTO statement VALUES set, so you've got to be careful when you want to use the data set not in the full contents. I should note that I had to rewrite all the software which generated the SQL statements for the proper (back-)quotation. This was a handful of complicated tasks of fixing the Bourne shell and Perl scripts. And I realize why SQL injection is so popular for attacking the database servers. Parsing SQL correctly is a non-trivial process. A word can be either a part of a directive or a target identifier, depending on the position where it is in an SQL statement. So when you want to store CALLs into a MySQL database, you've got to do it carefully with (back-)quotation. Data erasing woes 2008-02-04T17:07:00.000+09:00 Clearing up the used disk has been a real burden for sysadmins, as exploitation of carelessly-discarded information becomes popular. In Japan, people are extremely sensitive against how their personal data records are kept secret and controlled. The reality is, however, that data disclosure incidents keep happening, no matter the government enforces the law, and companies make frequent apologies for those incidents. Once the data get out, you won't be able to take them back. I think a part of the reason of continuing data disclosure incidents is that erasing data is simply a difficult task. For example, even if you store only 10G bytes into a 160G-byte disk, you need to thoroughly sweep the whole 160G bytes, to guarantee all data are cleared. Also, you've got to be careful to deal with the hidden data, which is not accessible by the data erasing software. If you really want to make the data not recoverable at all, you need to erase the trace of electromagnetic residue on the hard disk platters, which is hard to perform in a usual business or office environment. Destroying a usable device is not an environmentally-friendly practice either. Another problem is that the sweeping process is slow. The physical writing speed governs the whole performance. For example, one of my old portable 40G-byte hard disk can be written in only 20M bytes per second. So it will take at least 2000 seconds or about 34 minutes to sweep out the whole disk once. You need to do this multiple times to ensure the residue of data is not easily detectable, so the whole process may take 2 or 3 hours. 3 hours for just 40G bytes. 12 hours for 160G bytes. (sigh) The data storage which an individual has to manage is getting bigger and bigger every year. I wonder how people can cope with this. Do you sweep and erase the used disk data before you resell or give it away to somebody? Simple access-control status display 2008-01-31T18:00:00.000+09:00 I had a chance to talk to my business partners about computer networking security. One of the participants told me that a secure networks should not be mixed up with the unsecure ones. He said in a trusted operating system environment, you would see a red window and a green window, when you handle the different security levels of the information in the windows. You can't copy-and-paste the pieces between the red and green windows, unless an overriding procedure is accepted, and recorded. While I think you can do something more to assist visually-handicapped people, this simple definition of security models should be incorporated with many Web and other Internet applications as soon as possible. A few years ago I had a chance to see a presentation of similar ideas [1], which described the difference between Microsoft and Symantec products on telling the security level of the Web pages to the users. I thought Symantec's simple color-based access control looked much better than the IE's massive pairs of (not necessarily trivial) parameters. Access control is often very complex, but the ideas should easily get through to the users. This simple principle has not been popular yet in the world of computer software and Internet applications. Related link: [1]Steven Furnell: Fostering the usability of information security solutions, at OECD-APEC Workshop on Security of Information Systems and Networks, Seoul, Korea, 6 September 2005. A speculation: SMTP may disappear from the global Internet soon 2008-01-29T20:45:00.000+09:00 Long time ago in the late 1990s, I was a system administrator, maintaining a bunch of SMTP (mail transfer agent) servers. Since 2007 I have quitted running SMTP servers at home. The number of SPAM messages have rapidly increased since then; 95% of my received mail messages are considered SPAM and unread now. I've been watching this on multiple mailboxes using different SPAM-filtering algorithms, and the trend is mostly the same. Only 5% of mail messages are meaningful to me, including those from my business partners and mailing lists. On a DNS operator meeting last year, I was joking to one of the participants that the current SMTP (email) and DNS will cease to exist much sooner than we expect, due to the complexity of the running systems. People tend to believe they are always up and running, but the reality is that a lot of sysadmins are killing themselves to get the systems up and running. I think DNS surely has a lot of economic incentives to keep it alive as it is now. For example, domain names are protected under intellectual property rights and the international treaties. So somebody will keep the system alive, although the implementation details may change. On the other hand, many people have already seen the limitation of email; they are not pervasive as they were a decade ago. People are fed up with SPAM, and try not to rely on email too much. Many of them have already been using other messaging systems, such as the one implemented in various SNSes. In other words, they put a lower priority to email in general. Some people say the easiest way to deal with SPAMmers is to constantly change the mail addresses, which I do not think so. I think SPAMmers nowadays exploit all the traceable resources of mail addresses, so the leading time you can gain by changing the email address is getting smaller and smaller every day. The hardest part to fix, and the impossible part to fix, is the promiscuity of SMTP, to accept everything without authentication. Many people have said this and I know this is a mere repetition, but it is a fact we must recognize. Recently I tend to feel more than ever that I really don't care about the medium or protocol, as long as I can stay connected to the people who I want to. Of course it might get harder to meet new unknown people and communities, but I think that role has already been replaced by open blogs and the update notifiers or RSS readers. The fall and decline of Mixi 2008-01-28T08:30:00.000+09:00 How Alexa finds the difference between mixi.jp, facebook.com, and myspace.com. You will see how the Japanese SNS giant Mixi is smaller than the rest of the world. Even worse, Mixi is losing the users, at least from the PC/Internet domain. Mixi's markering target is shifting towards the Keitai (cell phone) users, which will not show up on the Alexa pageview statistics. I think this is a bad change of tactics, at least for their business. The number of Keitai users (of cell-phone users in and only in Japan) has already been saturated and does not have much room to grow any more. If Mixi depends on the ad business, the revenue will decline. And I feel the emptiness of Mixi every day as one of the users; less number of diary articles, less frequent updates, and the persistent scanning by bots. It's getting more and more like a deserted city. New Year Resolution 2008 2008-01-27T17:58:00.000+09:00 Dear Kenji: this is your new year resolution for 2008. a) Love your job. Love computers, coding, experimenting, and hacking. b) Redesign your financial portfolio, and how to earn money. c) Re-learn abstraction and principles of Internet and electronics. d) Continue reducing home computer system; downsize and outsource. e) Optimize online activities; publicize your works and minimize rants. f) Do not waste time on DXing; spend time for more creative activities. g) Practice DJ and recordings; stop wasting time on thinking about music. h) Always STAY FIT; eat less, go out of your house and move around. i) Interact with real people; love your family, respect your friends, and nurture productive relationships.