Capistrano 3: Change SSH port from default for the first time


We usually change SSH port to different one for some security. Like when provisioning server from official Ubuntu AMI, connecting to a server fails using same ssh_config without specifying port 22 explicitly.

the following quick hack task adds Port line on /etc/ssh/sshd_config then restart sshd. This works on Ubuntu 14.04 trusty. Change ensure_cmd for your system. Note that this adds listening port, not replacing port. More modification on sshd_config will be done via provisioning tool which run after this simple task, so this task only does simple one.

I recommend to make this task runs before deploy task.

task :ensure_ssh_port do
  on roles(:app) do |srv|
    user = srv.ssh_options[:user]
    port = srv.ssh_options[:port] || Net::SSH::Config.for(srv.to_s)[:port]
    unless port
      puts "ensure_ssh_port(#{srv}, #{port}): skip"

    puts "ensure_ssh_port(#{srv}, #{port}): start"

    user_opt = user ? "#{user}@" : ""

    if system(*%W(ssh -T -p #{port} #{user_opt}#{srv} true), err: File::NULL, out: File::NULL)
      puts "ensure_ssh_port(#{srv}, #{port}): ok"
      execute "echo '#{srv} port ensured'"

    unless system(*%W(ssh -T -p 22 #{user_opt}#{srv} true), err: File::NULL, out: File::NULL)
      abort "Couldn't connect #{user_opt}#{srv} with both port 22 and #{port}"

    puts "ensure_ssh_port(#{srv}, #{port}): port 22 ok, changing sshd"

    ensure_cmd = "ssh -T -p 22 #{user_opt}#{srv} \"sudo sh -c 'echo Port #{port} >> /etc/ssh/sshd_config && service ssh restart'\""
    puts "ensure_ssh_port(#{srv}, #{port}): $ #{ensure_cmd}"
    system(ensure_cmd) or raise 'failed to ensure'

    execute "echo '#{srv} port ensured'"
Published at 2015-07-23 03:23:04 +0000 | Permalink

Building AMI from scratch using packer amazon-ebs builder


HashiCorp's Packer is a useful tool to build some VM images for multiple platforms and softwares. Using builders like virtualbox-iso allows building images from scratch; installing systems into empty disk. It supports AWS EC2 AMI, but it doesn't allow building from scratch straightly.

So I've discovered the following 2 ways to build AMI from scratch using packer:

Plan A: use customized builder amazon-scratch

First I developed https://github.com/sorah/packer-builder-amazon-scratch . This allows attaching additional disk on source instance, then creates AMI from additional disk. This works well but this can't be used on Atlas, because it can't install any plugins.

Plan B: boot from tmpfs using user_data

amazon-ebs builder supports user_data for source instance. Ubuntu images have cloud-init that do some initialization process using metadata including user_data.

This plan uses amazon-ebs builder with source EC2 instance booted from tmpfs. The following cloud-config bootcmd injects bash script before /sbin/init run, then reboot instance itself. The script copies all of rootfs into tmpfs, then unmount root EBS, finally kicks /sbin/init on tmpfs to continue boot process. So provisioners can format /dev/xvda and install systems into it.

Also this scripts change sshd listening port from 22 to 122 -- to make sure packer to connect instance after reboot. You have to specify ssh_port to 22 on your packer configuration.

I'm using this trick on https://github.com/sorah/gentoo-build -- this works well for packer, out-of-the-box.

Published at 2015-05-11 03:11:28 +0000 | Permalink

OS X: Determine power source is connected or not in command line

Use ioreg :

ioreg -rc "AppleSmartBattery" |grep ExternalConnected|awk '{print $3}' | grep -q '^Yes$'

(see exit code)

Today accidentally our MacBook for Jenkins Slave has down -- due to power source loss. So I've added cron job to notify my team that power cable is disconnected based on this one liner.


Published at 2015-04-23 00:09:33 +0000 | Permalink

Monitoring fluentd with zabbix


Fluentd has monitor_agent plugin to expose its plugin status (buffer, queue, etc) via HTTP API: http://docs.fluentd.org/articles/monitoring

  type monitor_agent
  port 24220

By using this you can monitor fluentd buffer information with Zabbix user-defined discovery.

require 'json'
require 'open-uri'

PLUGINS_URL = "http://localhost:24220/api/plugins.json"

json = JSON.parse(open(PLUGINS_URL, 'r', &:read))

  data: json['plugins'].map do |plugin|
      "{#PLUGIN_ID}" => plugin['plugin_id'],
      "{#PLUGIN_CATEGORY}" => plugin['plugin_category'],
      "{#PLUGIN_TYPE}" => plugin['type'],

place this in favorite location (at here /usr/bin/fluentd-zabbix-discovery,) and define the user parameters:

UserParameter=fluentd.plugin.retry_count[*],curl -s localhost:24220/api/plugins.json| jq -r '.plugins[] | select(.plugin_id == "$1") | .retry_count'
UserParameter=fluentd.plugin.buffer_total_queued_size[*],curl -s localhost:24220/api/plugins.json| jq -r '.plugins[] | select(.plugin_id == "$1") | .buffer_total_queued_size'
UserParameter=fluentd.plugin.buffer_queue_length[*],curl -s localhost:24220/api/plugins.json| jq -r '.plugins[] | select(.plugin_id == "$1") | .buffer_queue_length'
UserParameter=fluentd.plugin.type[*],curl -s localhost:24220/api/plugins.json| jq -r '.plugins[] | select(.plugin_id == "$1") | .type'
UserParameter=fluentd.plugin.plugin_category[*],curl -s localhost:24220/api/plugins.json| jq -r '.plugins[] | select(.plugin_id == "$1") | .plugin_category'
UserParameter=fluentd.plugin.plugin_id[*],curl -s localhost:24220/api/plugins.json| jq -r '.plugins[] | select(.plugin_id == "$1") | .plugin_id'

Then you can define template like this: https://gist.github.com/sorah/cfbb39cb750f9bdbdeb2

Note that this plugin creates item using plugin_id, so defining proper plugin_id in fluentd's configuration is highly recommended.

  @id my_favorite_input
  type something
<match **>
  @id my_awesome_output
  type something
Published at 2015-02-01 15:06:22 +0000 | Permalink

Running gocode under dependency manager

nsf/gocode searches *.a object files under $GOPATH/pkg/$GOOS_$GOARCH by default. But dependency manager for Go customizes $GOPATH for its environment on build, so gocode can't find object files under dependency managers, such as godeps, gondler, or gom.

To make working correctly, set lib-path for those tool. Godeps workspace is usually placed at Godeps/_workspace/pkg/$GOOS_$GOARCH. Relative path works well.


gocode set lib-path 'Godeps/_workspace/pkg/linux_amd64:_output/local/go/pkg/linux_amd64'
Published at 2015-01-21 01:21:25 +0000 | Permalink

Restricting traffic to kube-proxy only from trusted networks


As of kubernetes 0.8.0, there's no official way to restrict traffic to kube-proxy.

I'm using the following iptables rule to restrict traffic only from local network and docker containers.

# These rules should be before `-j KUBE-PORTALS-CONTAINER` and `-j KUBE-PORTALS-HOST`
-t nat -A PREROUTING -i docker0 -d YOUR_PORTAL_NET j MARK --set-mark 8820
-t nat -A PREROUTING -s YOUR_LOCAL_NET -d YOUR_PORTAL_NET -j MARK --set-mark 8820
-t nat -A OUTPUT -s YOUR_LOCAL_NET -d YOUR_PORTAL_NET -j MARK --set-mark 8820
# Allow marked packets
-A INPUT -i docker0 -m mark --mark YOUR_FAVORITE_MARK -j ACCEPT

Replace YOUR_LOCAL_NET with your local network (e.g., and YOUR_PORTAL_NET with your kube-apiserver's -portal_net configuration.

Published at 2015-01-11 05:22:53 +0000 | Permalink

Hash#reject regression in Ruby 2.1.1

In Ruby 2.1.0 or earlier, the reject method in any class that inherits Hash returns an object of its own class. However, in Ruby 2.1.1 this behavior has changed accidentally to return a plain Hash object, not of the inherited class.

class SubHash < Hash; end
p SubHash.new.reject{}.class #=> 2.1.0: SubHash 2.1.1: Hash
p Hash.new.reject{}.class #=> 2.1.0: Hash 2.1.1: Hash

(To be exact, extra states such as instance variables, etc aren't copied either. With the release of Ruby 2.1.0 we've changed our version policy, so 2.1.1 shouldn't include such behavior change. )

This regression could potentially affect many libraries, one such case is Rails' HashWithIndifferentAccess and OrderedHash. They are broken; as the reject method now returns a plain Hash instead of HashWithIndifferentAccess or OrderedHash. https://github.com/rails/rails/issues/14188

Why is this happened

Firstly, this is not an expected change. It's an accident due to one missing backport commit into 2.1.1.

This behavior change was originally discussed in bugs.r-l.o#9223. However, it had been rejected for the release of 2.1.0 because it was too late. So, this change was rescheduled for Ruby 2.2.0, with a deprecation warning added to 2.1.0.

Commits around this change are described here, read the following gist for more detail: https://gist.github.com/sorah/9265008

Ruby 2.1.0 contains a constant in C to switch Hash#reject behavior by using #ifdef. Hash#reject will return a plain Hash by setting this C constant to 0. When this constant is set to 1, Hash#reject will return the object of its class with any extra states.

After checking out the 2.1 branch, the revision 44358 changed this constant name, and was backported to the 2.1 branch. However, this commit had leaked one line which changed the constant name. This leak is fixed in revision 44370, but this was not included in the backport to the 2.1 branch. Yes, this is reason of the regression.


So I recommend to build a patched Ruby 2.1.1 with revision 44370 or add this monkey patch to your application: https://github.com/rails/rails/pull/14198/files

As of now, revision 44370 is backported to the 2.1 branch, so this accidental behavior change will be fixed when Ruby 2.1.2 is released. https://bugs.ruby-lang.org/issues/9576

As I wrote above, this behavior change is still scheduled for the release of Ruby 2.2.0. I recommend to fix your code to in order to expect this behavior change. One option is to re-define the reject method to your class like Rails pull#14198 does.

(This article is translation of my Japanese blog with request and help from zzak <3. Thank you for proofreading, zzak!: http://diary.sorah.jp/2014/02/28/ruby211-hash-reject)

Published at 2014-03-10 13:50:01 +0000 | Permalink

render_to_string doesn't work well in ActionController::Live

render_to_string doesn't work well in ActionController::Live.

Because render_to_string modifies response_body and restores it. But #response_body= regenerates response.stream and ActionController::Live's overridden #response_body= closes response.stream, so response.stream.write won't work after any render_to_string in ActionController::Live.

def render_to_string(*)
  orig_stream = response.stream
  if orig_stream
    response.instance_variable_set(:@stream, orig_stream)

Above code works well as monkey patch to fix this issue.

I requested to pull this to upstream: https://github.com/rails/rails/pull/11623

Published at 2013-07-28 00:54:55 +0000 | Permalink

Class Variables and Instance Variables on Class, in Ruby

Do you know problems around class variables in Ruby?

Class variable

You can declare class variables by using @@ for prefix of variable name, for instance: @@foo.


But, class variables can easily overwrite by subclasses. This is based on Ruby specification; class variables can be shared on its subclass.

Class variables are similar with global variables. They're too hard to handle safely.

For usually cases, I can't recommend to use.

Declare Instance Variable on Class object

So then, how we define "class variable," safely?

In Ruby, classes are object. This means you can define instance variable on class.

Scope of instance variables on class are closed within the same class. Thus, they don't effect on subclasses.

(Of course you can use attr_accessor, attr_reader, attr_writer. Example code)

Using from instance

Here are how to use that variables from instance objects.

Use attr_accessor

The simple solution.

Use instance_variable_get

but if you wanted to protect from foreigns, you can use instance_variable_get and private method.

Published at 2013-01-29 02:29:02 +0000 | Permalink

Run rbenv in global

Updated (Feb 13, 2014): Fixed my poor English. And note that I hadn't encountered problem around my global rbenv installation for a year.

When we want to have ruby in a server, sometime we can't use the system's package manager.

Because package managers (in almost distros) serve older Ruby. Plus, they may have weird complex rubygems integration.

So the following are my ideas to get Ruby in your servers:

  • Build packages by ourselves: deb, rpm, ebuild

    • I think helpful when you have many servers, but when not, it might be overkill.
    • Building and maintaining packages are boring stuff.
  • Install ruby by hand: make && make install

    • Difficult to control later. What you do when you want to uninstall ruby?
    • How to upgrade cleanly?
  • Version Managers: Install rbenv, rvm

    • I think this is the bet solution; at least for me.
    • I prefer rbenv because it's made simple. rvm changes many shell behaviors.
    • rbenv is great for installing globally.

So, I'm using rbenv for my servers.

But Ruby is used from many users (including system users, daemons). Normally we install rbenv in ~/.rbenv, but it's depends on user's shell ($PATH) configuration and user's home directory.

Here's how I installed rbenv in global.

Install rbenv and ruby-build

I used /usr/local/rbenv for place rbenv.

sudo -i

cd /usr/local
git clone https://github.com/sstephenson/rbenv rbenv
mkdir -p rbenv/plugins
git clone https://github.com/sstephenson/ruby-build rbenv/plugins/ruby-build


rbenv looks RBENV_ROOT for many paths. In default (= when not specified), ~/.rbenv is set.

When installed rbenv in /usr/local/rbenv, we should set it to RBENV_ROOT=/usr/local/rbenv.

Install ruby in rbenv by ruby-build

Simply do:

sudo env RBENV_ROOT=/usr/local/rbenv install 1.9.3-p374
sudo env RBENV_ROOT=/usr/local/rbenv rehash

Then, set installed ruby to default ruby:

sudo env RBENV_ROOT=/usr/local/rbenv global 1.9.3-p374

Okay, we've done setup ruby on /usr/local/rbenv/shims/ruby. but, still requires to set $PATH. Let's solve it.


I wrote simple plugin for rbenv: https://github.com/sorah/rbenv-global-rehash

This generates symbolic links and scripts from ${RBENV_ROOT}/shims and ${RBENV_ROOT}/bin into specified directory.

For ${RBENV_ROOT}/bin, this plugin generates script to invoke with suitable $RBENV_ROOT.

But scripts in ${RBENV_ROOT}/shims (generated by rbenv rehash) already contains export RBENV_ROOT, so this script generates symlink for shims.

Install this:

sudo git clone https://github.com/sorah/rbenv-global-rehash /usr/local/rbenv/plugins/rbenv-global-rehash


sudo env RBENV_ROOT=/usr/local/rbenv rbenv global-rehash /usr/local/bin

Finally you can:

/usr/local/bin/rbenv versions
/usr/local/bin/ruby -v

# RBENV_ROOT will not be required after the first run
sudo rbenv global-rehash /usr/local/bin


Published at 2013-01-17 22:56:13 +0000 | Permalink

Renewed this blog

Just released days.gem, the simple blog system built up with Sinatra, and migrated this blog from Lokka.

And, I'll never write Japanese-only article for this blog, please see also http://diary.sorah.jp/ for Japanese entries.


Days is simple blog system built up with Ruby + Sinatra.

This system's big feature is: core separated from deployment.

To set up days and start it, we have to do:

$ gem install days
$ mkdir my_blog
$ cd my_blog
$ days init
$ days migrate
$ days server

by days init, days.gem generates simple Gemfile, config.yml, and config.ru.

But you can customize appearances by placing haml views to views directory.

Core application part is included in days.gem, and it's required by config.ru and gem bundler.

I'm writing tutorial and documentation of days.gem. Stay tuned, please!

Published at 2013-01-04 05:27:50 +0000 | Permalink

Show warning for RSpec examples that has no expectation


You may sometimes forget to write should for matchers.

it { a == b } # ng
it { a.should == b } # ok

Using the above RSpec configuration, that warns for such cases.

$ rspec -fd spec/a_spec.rb
[WARN] No expectation in example at ./spec/a_spec.rb:4: You may forget to write `should` in the example
  example at ./spec/a_spec.rb:4
Published at 2012-12-17 10:33:05 +0000 | Permalink