Cloudsourcing from home

Managing the information and services which I provide has been a persistent source of headache since I set up my own Web server at home in 1997. So far I've been successfully managing the contents, such as:

• External authoritative DNS (though you still need to maintain the internal authoritative DNS and the cache servers)
• Mail receiving servers (or more formally called the message transfer agents or MTAs) and the complex forwarding schemes for my family, namely me and Kyoko
• Static web servers solely for the public contents, including this blog

I still have to pursue moving out the servers from home to the external sites, however, because managing new protocols from home such as IPv6 and other new applications is increasingly getting difficult, due to the constraints of available bandwidth, IP address space, and my own time.

Frankly speaking, I do not want to buy any more computer at home. More accurately, I do not want to install and maintain operating systems and the application software at home unless I really have to. Things have already been getting too complex, and I need to offset or outsource the complexity to external sites and services.

The problem I'm facing is that cloudsourcing or moving out services provided at my home servers adds many levels of indirection and layers of things to consider: availability and redundancy issues, association of the services, how and where to consistently collect the backup data, and the contingency plan and the recovery procedure in case of service disruption.

I know the cloudsourcing issue is a matter of business reengineering process, and to redesign the workflows of the various activities, in my family. It's not only about the computer systems. We need to change our way of living.

And I finally started to learn what kind of things you can do on Google Apps, Google AppEngine, and Amazon Web Services.

My new challenge in 2010: security issues at university network systems

Since April 2010, I've been working on the network security management of Kyoto University. There are many things to consider. I've already published a report in PDF about the status and issues.

The fundamental principle which always haunts me is that adding security to a not-so-secure system is far more difficult than making the security component built-in into the system. In a large organization, you cannot change the policies and configuration of the running systems overnight; you need to negotiate with a lot of stakeholders to reach a consensus or an agreement. Consolidation takes a lot of time, especially for a well-established system such as the email servers.

Another difficult issue to handle is how not to restrict users from the legitimate use of the campus systems. University is an organization of research and education. While leaving a system vulnerable for the known attack vectors is not an option anymore, allowing failures for learning from the try-and-error process should not be prohibited. Without a firm security policy, you cannot really decide what is right or wrong.

Catching up with new technologies is also a tough requirement to meet for a large network system. The global IPv4 address space will be used up in next year 2011, so introducing IPv6 is a must, though not urgent; and the organizational DNS subsystems should be DNSSEC-ready, for both the cache resolvers and authoritative servers. And I have to be able to explain those changes will surely benefit the users and are worth paying for.

I'm sure I'm going to face a lot of existing and new problems on next year 2011. And I'm hoping I will be able to solve at least some of them.

Writing for job seeking

For the past few months, my top priority issue has been searching for a new job. My current (as of March 2010) job contract will terminate by March 31, 2010. Looking for a job itself is very stressful, and you need to focus on writing a lot of documents, including your CV, cover letters, etc. In this article I will introduce a few Web sites which might help job seekers in general.

Ask A Manager, a blog written by a real human resource professional, has been particularly helpful during my job searching period. The articles on the blog suggest me that corporate and organizational cultures in Japan and the USA are much more similar than different with each other. I think the psychology of hiring and getting hired is highly universal.

For writing a CV, Dave Levin's job application page has a set of useful examples, especially for those searching for IT research/academic positions, with the statements of research and teaching. His energetic, upbeat and rational writing style, while not getting overly emotional, will help you find how to make your CV fit for the job openings. (Note: of course you need to change the details of your CV for each job you apply for.)

Getting the job contract terminated, laid off, or even fired, is itself a tedious and tough experience. I've read a few books for helping myself:

• Hugh MacLeod's Ignore Everybody: this book and his comic strips remind me of the hard cold facts that creative people must endure loneliness, and that they have to earn money before doing what they want to do.
• Martha I. Finney's Rebound is what ACM first suggested me for career development. The good thing about this book is that it starts from describing how to cope with the traumatic and angry feelings when you first find out you're getting pushed out of a job.
• Chad Fowler's The Passionate Programmer, formerly called My Job Went to India, describes how a software developer should cope with the change of trends and other social environmental factors. I highly recommend this book as a list of checkpoints for sorting out your experience and finding out what you need to do for doing what you really want to do.

And I suggest you to think carefully about how you expose yourself through blogs, Twitter, Facebook and other social media, during the transition period of your job. Your articles or written pieces on the Internet are public and not private; an Ask A Manager article about how managers feel about blogging of their employees represents a practical view of pros and cons of blogging in a plain English.

I also suggest you to make your past blogs private, when you feel awkward about what you have written there, even slightly. Then you can reopen them after editing and removing the articles before you get embarrassed.

Persistent Erlang processes or process pairs

Erlang's processes are the minimal unit of execution of Erlang BEAM virtual machine. Each process has its own ID, and can send/receive messages, register the name to the BEAM it's running, and can link with another process for error handling and monitoring. It even has its own process dictionary.

I've been thinking about a question for a few days: can you make a computer holding a process for more than 100 years? The word process here does not necessarily have to be the Erlang one, but the Erlang process will be a good candidate because the working environment it has to carry around with it is minimal and much smaller than that of UNIX process.

Keeping a process alive for a long time would not be possible if a process is confined to a single machine; the machine's failure means the immediate death of the process. So the process should be able to move around between multiple machines and make its clone on its own. Using shared memory should be avoided as possible. Realizing these characteristics with Erlang is less difficult than in other computer language systems.

I discovered the idea of having a persistent computer process is actually not my original. Google search engine tell me that Jim Gray has already published a technical report when he was in Tandem in 1985 (PDF copy from HP Labs) (a scanned text file of the report) with an idea of persistent process-pairs, as a part of his model of transactions for simple fault-tolerant execution.

In Gray's report, he describes a much smarter approach of making two persistent processes a pair to realize the persistency. If one of the pair fails, another one will show up and take over the actions of the failed one. This idea is much wiser than trying to keep a single process alive.

So now I find a way to realize persistent processes; next I need to learn how to implement them. It'll be a part of my new year's resolution for year 2010. A happy new year to you all the readers.

DNS operation is utterly neglected by many people

Twitter outage via DNS hijacking showed another case of common symptom: DNS operation is simply neglected by people doing business on the Internet.

I was doing research on DNS transport security from 2002 to 2008. One of the reason I quit focusing on the research was that most, if not all, of the DNS problems are caused by operation failures, not necessarily due to technical deficiency of the DNS protocols and systems. In short, it's too political and social to do the technological experiments over DNS.

I still think DNS transport protocol issues are critical for stable Internet operation. But solving those issues does not help recovering human errors, such as lame delegation (missing link) between the domain name hierarchy. And stable operation of DNS systems is very difficult to maintain without stable hardware, software, networks, and operators.

I notice many small companies (especially in Japan) keep their authoritative servers inside their office, which is not good from the stability point of view. Actually, for many small Internet sites, including mine, not so many DNS zone records have to be exposed to the public. So I've already outsourced the DNS authoritative servers, while I periodically watch whether those servers do the right thing.

DNS is by definition a distributed system; and the management standard is much lower than what people (and even Internet engineers) believe. For the further details of how DNS is not well-managed, I suggest you to read a more detailed commentary on how important DNS is as an asset, by Danny McPherson of Arbor Networks.

Bruce Schneier's speech at IWSEC2009

I had a chance to meet Bruce Schneier face-to-face for the first time, when I attended his invited talk session at IWSEC2009 conference in Toyama, Japan, on October 28, 2009.

I once worked for translating Schneier's book Email Security (published in 1995, which is now declared outdated by him) into Japanese. At that time he was a technologist on cryptography. The keynote speech in Toyama showed, however, that he was rather interested in psychology and human behavior, which is not necessarily logically predictable and often considered errorneous from technological points of view.

While I read a few people who apparently tweeted Schneier's speech was boring, I found his speech on the psycology of security rather refreshing and interesting. Maybe that's because I've been frequently disillusioned by how technological solutions often backfire. Of course it's not about the details in cryptography or other security protocols which are the primary topics of IWSEC so that might have been boring for the majority of the participants.

I won't go into the details of Schneier's speech, because most of the individual topics are frequently covered in his blog. Let me write about one of the things intrigued me the most; it was about the risk heuristics. People are risk-aversed or trying to have sure gain. And at the same time, they prefer probabilistic loss or risk-taking behavior when they have possibilities of losing something. With this heurisric way of thinking, people usually don't want to pay for having less risky life, and this is exactly one of the reasons why security products don't make good sales.

After the speech, I asked him why he converted from pure technologist to rather a scientist of broader topics including psycology and sociology. Unfortunately I didn't get a definitive answer on what made him so; he only emphasized the sociological aspects of security were equally important and critical as the technological ones. Maybe I could find the answer in one of his books; especially if the reason is a highly personal one, which no one will ever know.

Erlang and Github

Erlang/OTP is now officially maintained under the Github repository, since the release R13B03. I think this is a milestone for the language, because the Ericsson development team finally decided to show the interim results of what they are doing for the time being.

One of the characteristics I like about Erlang is that the language specification and libraries have been maintained by a single entity called Ericsson's Erlang/OTP Development Team. I do not want an anarchy for computer language and operating systems. I prefer BSDism than Linuxism in this sense; I think pieces of code should be rather controlled by the core people while sufficiently accepting improvements from the other developers.

The old Erlang/OTP daily snap archives, however, are no longer sufficient to catch up with the daily development cycles. And many non-Ericsson authors have put in their patches into Erlang, including mine. So there had to be some systems to accept user feedbacks.

Using an open repository system such as Github is a wise idea for incorporating new code into Erlang/OTP, and showing the official status of modifications. Git is flexible enough to allow per-user and per-purpose branches. And Github allows forking between the users. The Ericsson's Team doesn't have to build and publicize its own code repository system for Erlang/OTP, which will cost them significant amount of human and financial resources.

And now I have an official requirement to learn Git; to catch up with the Erlang/OTP development cycles.