Marunage - a dark side of omakase

If Rails is omakase, Japanese construction and IT projects are marunage.

When I first heard about the popular readout and the written article of Rails is Omakase, I immediately thought about another common Japanese word called marunage 丸投げ (pronounced mah-rue-nah-gay), which literally means passing everything onto someone else and completely getting rid of the things which you passed onto.

The actual usage of the word marunage is more vicious and exploiting; large corporations and government organizations often do the marunage to their contractors. When you do the marunage of a project, it means you are getting rid of all the responsibilities and liabilities from the project and letting someone else take care of the mess and the aftermath. In this sense, marunage is a form of risk dumping, or simply a bad form of delegation. The analogy of marunage is also quite common for the national projects. For example, the ministries do the marunage to the research subsidiaries.

I believe the word marunage originally comes from the construction industry, which is very popular in Japan for multi-level delegation from a large general contractor aka zenekon ゼネコン (pronounced zeh-neh-con) to the smaller and powerless contractors aka shitauke 下請け (pronounced shi-tah-ooh-keh). The business model here is that the shitauke companies do the most if not all the actual tasks, and the zenecon does little things other than taking brokerage fees. Multi-level sub-contracts are also popular; for example, in Fukushima Dai-ichi Nuclear Power Plant cleanup project after the meltdown incident on March 2011, ten levels of sub-contracts were revealed.

The mindset of zenekons enslaving the shitauke companies is also common in IT industry in Japan. The IT zenekons, also known as shisutemu integureitaa ("systems integrator" aka "SIer"), have been historically accused of making questionable practices on winning large-scale project contract bids, such as winning a contract by proposing unrealistically budget. For example, in 1989, there was a case of major SIer repetitively tried to win bids by proposing one (1) yen (JPY) as the budget for a whole project of municipal governmental systems. Also, those large SIers or IT zenekons maintain strong relationships with the large-scale customers, and gain profits by doing the marunage of the projects to the shitauke companies. I won't dig deeper in this dark side of Japanese IT history here, but you may find many similar cases.

Getting back to the definition of marunage, I believe that the mindset of marunage represents the feverish risk-aversion towards everything among Japanese people. Still many if not most workers in Japan think a single failure means the professional death which cannot be recovered later, and the legal and economic systems are very harsh against people who have the failure records, for example, a bankruptcy. This kind of culture forces the decision makers to do the marunage to avoid risks. And I see the culture of marunage shows how Japan is more and more closely like the former Soviet Union, one of the most bureaucratic and stagnated nations, disbanded by the economic failure.

So when you hear about marunage, you've got to be very careful; don't become the victim, and try your best to avoid becoming an oppressor by doing the marunage to the powerless subordinates, colleagues, coworkers, or even your family members or bosses.

Computing with only one eye

My right eye has not been really functioning since 1982, when I was 17.

I wore cataract first in May 1982 while I frequently hit my right eye due to the strong itchiness caused by chronic skin allergy. Eventually I lost my sight from the right eye, and after a few eye check the retina detachment was discovered. I was immediately thrown into a hospital room, locked onto the bed with keeping my face up against the gravity (so that the gravitational force helped preventing further detachment) for three weeks, in June 1982. It was an awful experience for a young teenage boy.

I had to have another surgery on September 1982 because of the unfixed detachment again. Later I had another surgery to fix the squint in 1983, caused by the previous surgeries. And finally in March 1986, the swollen and white-colored lens cell (yes, the lens is a single cell) which had lost the transparency in my right eye finally got taken away.

Fortunately, the retina (not the Apple's high-resolution display) in my right eye is mostly functioning OK, though I really can't read anything with it, and I can no longer accurately identify the 3D depth any more. I usually ignore the sight coming from my right eye other than the level of the light.

I've been with the usability problem caused by the broken right eye for many years, long before I started professional programming. Fortunately I've learned how things get large and small when their distance changes, so I can sway to avoid getting hit from most of the things. I still can enjoy 3D movies, though I won't accurately focus anymore.

Things coming towards me from the right side, however, are hard to recognize. I've stopped operating any vehicles including bicycles; I don't have a driver's license either. You don't want to get into a trouble which you can easily expect, especially where the driver's seat is on the left side of the car, like in the USA or Canada. Japan is not, but that doesn't alleviate the problem.

During the age of 80x25 character terminals when DEC VT100 or VT220 terminals were popular in 1980s, my right eye disability didn't cause much trouble, because the whole screen fit very well in the visual space I could recognize. Even in the age of where 14" or 17" CRT or LCD displays, where the default aspect radio was 4:3 or 5:4, I didn't have much problem. I've written a lot of document and many pieces of code; fortunately none of them mandated me to tackle the three-dimensional loss of perception.

I have a few problems regarding my sight disability since the beginning of the 21st century, unfortunately. One of them is the color design; most web sites choose white as the background color, which interferes the black foreground characters I read. I try to choose darker background colors and brighter foreground colors as possible, but not all web applications allow to do so.

Another one of the 21st century problems is the change of default aspect ratio of the screen to 16:9; this is too wide for my rather narrow sight space. If you are watching a movie this wouldn't be a problem. I welcome high-resolution displays with crispier pixels and accurate positions. But when you write a text, this will be a serious problem, because you can't see everything at once. I always choose narrower horizontal size to keep the screen in my left eye's sight range.

So far I can manage the environmental change of computing in the past 31 years. I expect, however, the future computing models, including those require wearing glasses or assume three-dimensional recognition ability, may eventually reject me from the cutting edge of computing. Maybe I can ask Google to make a special Google Glass to compensate the loss of my right eye's lens and focus directly into my right eye's retina, with proper size adjustment so that I can see the things in two eyes again.

And I can assure everyone that you can write the code with sight disabilities; I know a professional programmer who has completely lost the sight too.

(Thanks Tyler Hannan, who suggested me to write this. I had been hesitating to write about my weakness on the web, but I thought this would be helpful to others.)

I've joined Basho Japan from February 2013

I've started my new career as a Senior Software Engineer at Basho Japan KK, a 100% subsidiary of Basho Technologies, Inc., effective 1-FEB-2013. I'm very happy that my colleagues are all very much supportive for my startup as a new developer. I will try my best to catch up the lean software development process and to get the most out of both the elementary and advanced technologies for contributing to the company.

Basho is a wolf pack of the bleeding-edge Erlang/OTP and distributed system engineers, who have made Riak as an open source standard of highly-distributed and fault-tolerant databases. Catching up with the skilled engineers is a really tough job, but it's also very exciting to take the new challenges for making the actual software products to the customers. This is completely different from what I've experienced in the past 12 years as an academic/government researcher.

Basho's customers have a lot of demands for Riak already, and the company including myself has to meet the expectation while shipping the products on time and maintaining the high product standard. I will contribute my 22-year skills of network engineer, sysadmin, and an OS developer to the software development process, in the most effective way.

My participation to Erlang/OTP activities and ACM Erlang Workshop will remain the same.

So stay tuned for my pull requests and pieces of code!

Lost in transportation

I will resign from my full professor position at Kyoto University. The employment contract will terminate by 31-JAN-2013.

I've got killed by commuting, again. Yes, again. I should have learned from my past dreadful experience from 1990 to 1992. I know regular long-distance commuting is hazardous to my health. And being forced to synchronously work with other people from 9am to 5pm simply does not work for me. But I had to take the job for my paycheck, because no other practical job offer was available on Spring 2010. And I have to admit that I did not survive the tough working condition.

I was taking a sick leave from November 2012 to January 2013. I got mentally crashed due to chronic and accumulated fatigue, presumably caused by spending nearly four hours every day during commuting from home. Fortunately at the end of January 2013 my mental and physical health conditions have been back to the normal ones, but at the end of October 2012 I was very weak and mentally disorganized.

I once experienced a similar situation of chronic fatigue when I was working in Tokyo, commuting to a Yokohama office, with four train line transits every day, taking nearly two hours for each direction, from 1990 to 1992. I was young enough to tough it out for two years, but I collapsed on Autumn 1991. I had to quit the job on April 1992.

Commuting from Northern Osaka including Toyonaka to Kyoto is relatively less stressful than commuting from Setagaya in Tokyo to Hodogaya in Yokohama. It takes three train lines for each direction, much less crowded than that of Tokyo. I have to walk a lot though in Kyoto and Osaka, so the time I have to consume is approximately two hours for each direction, virtually the same as the one I experienced in Tokyo.

I tried to reduce commuting by doing my job from home, but the nature of my missions eventually changed to mandate daily commuting, and my physical and mental health conditions got deteriorated. What was worse was that I also had to work from home. I was simply spending too much time on my working hours, and in the end my productivity turned into nil.

I took my personal records of working hours spent on each task at the workplace simply for my own curiousity, measured by the unit of Pomodori, which is plural of Pomodoro. Each Pomodoro in my case means 30 minutes. I take this definition of Pomodoro/Pomodori from a popular book called Pomodoro Technique Illustrated, by Steffan Nöteberg. Note that Steffan suggests that you have to take at least a five-minute break for each Pomodoro, which takes 25 minutes. This is unfortunately virtually impossible in the corporate life.

I think the worst time wasting events at the workplace were regular meetings. Many of the meetings take at least two Pomodori, and some of them even take six or more, which means three hours or more. Locked onto a chair for such a long time without a break makes you feel very sick, and will surely hurt your back. I also have to write that commuting only for attending face-to-face meetings is simply a waste of time and the physical energy. And I've found that for many days I had to work more than 20 Pomodori at the workplace, which suggested lower productivity.

I think allowing telecommuting, teleworking, working from home, or whatever frees the employees or managers from the physical office or being forcefully bound onto an unnecessary face-to-face meeting, is a must for a modern organization to boost the productivity of each organizational member. Some people may argue this is not for everyone, but I strongly believe this option should be available for all people, unless the person deliberately want to dedicate a fixed window of time to a fixed place with prior consent. I also think that the number of meetings should be minimized, and that the time spent on the dedicated meetings should be kept as small as possible.

I understand that occasional or sometime less frequent (for example per month or bi-weekly) business trips will be necessary, and I usually accept them. Face-to-face meetings are indeed necessary for some decision makers, which I also consider logical. But I think daily long-distance (more than 30 minutes or even one hour) commuting should not be used as a piece of litmus paper test of the royalty to an organization.

Unfortunately people at most of the organizations in Japan still think physical commuting and daily or weekly face-to-face meetings are critical and mandatory, and that there's no other way. I think this sort of thinking will no longer work well, regarding the increasing population density and train network failure rate, especially in Tokyo and the Kanto plain, and also in Osaka/Kansai, Nagoya, and less crowded regions. And people in Japan are now struggling to allocate their scarce resource of time to taking care of their family members, of their children, or of their elderly parents, which no one else will take it as the primary role. The time and physical energy management gets even more important for each individual than ever.

So I've decided to choose another way of living, before getting lost in transportation. Getting back to the days of working from home and telecommuting/teleworking is nothing new to me, because that's what I've done from 1992 to 2010. Leaving the career track of academic research is sad. But I need to live and survive. And I want to use my time and physical energy wisely. I'll take this new challenge again.

Coming change in 2013

(Photo by Kenji Rikitake, with a book cover of "Concurrent Programming in Erlang", which is an on-demand printed version via amazon.co.uk)

I've been away from this blog for quite a long time. This will be a short entry, though I'm writing this to make a mark at the beginning of the year 2013.

I'll be on a big transition in this year 2013. I'll write about the details when available. I can promise that I'll keep focusing on Erlang/OTP and distributed systems, as I have been for the past 5 years. Stay tuned for the details.

Lots of things happened last year. A good news is that I joined the 11th ACM SIGPLAN Erlang Workshop held at Copenhagen on September 2012. Meeting a lot of great people and nice friends there was a precious experience.

A bad news is that I have been taking a long sick leave since November 2012. I will write about the details later when I feel I should, but for the time being I can only conclude that a long-distance commuting literally killed me again, as it had done during 1990 to 1992. I definitely have to change my lifestyle.

On new year's resolution: I won't declare it this year, because I need to drastically restructure my professional life anyway. Of course I've got to get myself into a good shape. And I need to spend more time on code writing and development, which I could not for the past three years.

I hope things will work out in a good way this year; it's a year of Snake in Japanese/Chinese 12-year holoscope, and it's my year too.

Why Erlang people need the Erlang Factory events

Disclaimer: I am writing this blog article on my way home from SFO. The reduced air pressure in the passenger cabin might have affected the contents of this article.

I spent all three days of Erlang Factory SF Bay 2012, with two more days to cope with the jet lag. This was my third consecutive time to participate in the event as an invited speaker.

One of the most impressive things about this conference is that Francesco Cesarini, the leader of this event and CTO of the hosting company Erlang Solutions, carefully treats all delegates (which stands for the participants) equally with kindness and hospitality. Francesco's principles are well understood by all the other Erlang Solutions staff members as well.

I think Erlang Factory provides integrated feelings of satisfaction to all the delegated by letting them exchange the serious but friendly discussions. This is not something you can find over teleconferences or any forms of distant communication either synchronous or asynchronous. Erlang Factory is not a decision-making event; it is about sharing ideas and hanging out with each other, supporting both mental and emotional needs of the delegates.

Erlang Factory conference accepts a large number of versatile topics including:

• Distributed databases
• Case studies in production systems
• Detailed debugging into the library and the host operating systems
• Crash courses for programming and testing the language system, and
• Official announcements from Ericsson's OTP Team, who owns the final responsibility of managing and directing the language system's future.

An opportunity to discuss various complex issues with the speakers and the audiences is itself rare and precious, especially when the issues share the same core interests: Erlang/OTP.

Erlang Factory also accepts non-mainstream topics as a talk proposal. In the past three years, I talked about RPC over SSH, Mersenne Twister implementation with the NIFs, and in this year 2012 it was about IPv6 readiness and programming tips of the language system. Those are not necessarily the mainstream topics in the Erlang/OTP community, though I find quite a few problem reports on the online discussion places, such as in the erlang-questions mailing list.

My style of talk at the Erlang Factory is nothing extraordinary:

• First I analyze the problem and define the detailed problem domain to solve;
• then I write some example code to solve them and put them on the Web including GitHub and the other sharing places before finalizing the talk so that I could make it better with the wisdoms of others;
• I also experiment and evaluate the results while I'm finalizing the code and the presentation materials;
• Then at the Erlang Factory talks I discuss the conclusions found and propose the unresolved issues which may become another interesting talk topic.

The important points are sticking to the facts, no exaggeration, distinguishing assumptions from the facts, and listening to what the audiences ask and comment. Actually this is the same procedure I will take for the ACM Erlang Workshops, but for the Erlang Factory I try to make the presentation including more practical aspects and open questions.

I usually assign three months as a part-time project for an Erlang Factory presentation. The toughest part is to decide what to talk about. It's actually Francesco who suggested me to give a talk about IPv6 for this year 2012. I had some experience dealt with the technical issues of IPv6, so I decided to take the suggestion. The important point is that what you are going to talk is negotiable with the Erlang Factory staff.

This year 2012's events of Erlang Factory SF Bay Area were very intense, both in formal and casual senses. I've got an impression that it has become a truly international-class conference, while retaining the casual and friendly style of the past conferences.

I believe people go for conferences and events to meet people and share the feelings. It's a sort of festival, or Matsuri in Japanese, though also with the practical exchange of ideas. I strongly suggest all the Erlang/OTP enthusiasts to participate in this superb Matsuri.

Erlang VM memory size and ERL_MAX_PORTS

Erlang VM has the ports to communicate with other programs and objects (files, drivers, etc.) The ports are seen as the Erlang processes. The OS environment variable ERL_MAX_PORTS sets the default value of maximum number of ports. (See the manual for the function erlang:open_port/2 for the further details.)

I discovered by accident that Erlang VM of R15B on FreeBSD/amd64 9.0 in my test environment sets the value of maximum file descriptors assigned to the OS process as the default value for ERL_MAX_PORTS. This means ERL_MAX_PORTS is set to 800000, far larger than 1024, specified in the manual. This excessive value of the ERL_MAX_PORTS leads into unnecessary memory consumption of the Erlang VM. An easy way to test this is to execute the following code on the shell:

env ERL_MAX_PORTS=[given value] erl -noinput -eval 'io:format("~p~n",[erlang:memory(system)]).' -s erlang halt

How memory consumption changes just after the Erlang VM startup, as {ERL_MAX_PORTS, memory size in bytes}: {1024, ~65M}, {102400, ~124M}, {800000, ~543M}.

So I decided to set the ERL_MAX_PORTS for my home Yaws server to 4096. This significantly reduced the memory consumption.

I suggest you to tune the value of ERL_MAX_PORTS to optimize the number of concurrently opening ports of an Erlang VM, especially when you run a server on a memory-restricted host.

Update 11-MAR-2013: on R16B, the amount of memory consumed by the ports will be significantly reduced at the Erlang VM startup.

References

• Manual of erlang:memory/0
• [erlang-questions] About Environment variable ERL_MAX_PORTS
• [erlang-questions] Max open files and erlang memory