2014-03-10

Hash#reject regression in Ruby 2.1.1

In Ruby 2.1.0 or earlier, the reject method in any class that inherits Hash returns an object of its own class. However, in Ruby 2.1.1 this behavior has changed accidentally to return a plain Hash object, not of the inherited class.

class SubHash < Hash; end
p SubHash.new.reject{}.class #=> 2.1.0: SubHash 2.1.1: Hash
p Hash.new.reject{}.class #=> 2.1.0: Hash 2.1.1: Hash

(To be exact, extra states such as instance variables, etc aren't copied either. With the release of Ruby 2.1.0 we've changed our version policy, so 2.1.1 shouldn't include such behavior change. )

This regression could potentially affect many libraries, one such case is Rails' HashWithIndifferentAccess and OrderedHash. They are broken; as the reject method now returns a plain Hash instead of HashWithIndifferentAccess or OrderedHash. https://github.com/rails/rails/issues/14188

Why is this happened

Firstly, this is not an expected change. It's an accident due to one missing backport commit into 2.1.1.

This behavior change was originally discussed in bugs.r-l.o#9223. However, it had been rejected for the release of 2.1.0 because it was too late. So, this change was rescheduled for Ruby 2.2.0, with a deprecation warning added to 2.1.0.

Commits around this change are described here, read the following gist for more detail: https://gist.github.com/sorah/9265008

Ruby 2.1.0 contains a constant in C to switch Hash#reject behavior by using #ifdef. Hash#reject will return a plain Hash by setting this C constant to 0. When this constant is set to 1, Hash#reject will return the object of its class with any extra states.

After checking out the 2.1 branch, the revision 44358 changed this constant name, and was backported to the 2.1 branch. However, this commit had leaked one line which changed the constant name. This leak is fixed in revision 44370, but this was not included in the backport to the 2.1 branch. Yes, this is reason of the regression.

Accident.

So I recommend to build a patched Ruby 2.1.1 with revision 44370 or add this monkey patch to your application: https://github.com/rails/rails/pull/14198/files

As of now, revision 44370 is backported to the 2.1 branch, so this accidental behavior change will be fixed when Ruby 2.1.2 is released. https://bugs.ruby-lang.org/issues/9576

As I wrote above, this behavior change is still scheduled for the release of Ruby 2.2.0. I recommend to fix your code to in order to expect this behavior change. One option is to re-define the reject method to your class like Rails pull#14198 does.

(This article is translation of my Japanese blog with request and help from zzak <3. Thank you for proofreading, zzak!: http://diary.sorah.jp/2014/02/28/ruby211-hash-reject)

Published at 2014-03-10 13:50:01 +0900 | Permalink
2014-01-18

Photography workflow (2014 January)

Here's my current setup and workflow to take, develop, and upload photos.

2014 年 1 月現在の写真をとったときのアップロードするまでのフローです。

Equipments

I won't explain about camera itself in this article.

カメラについてこの記事ではこれ以上触れません。

Importing

  • Mount SD card on Mac
  • Move RAW+JPEG photos to ~/Pictures/YYYYMMDD_EVENT-NAME
  • Upload to local file server via SCP and Amazon S3 using Transmit.app.

S3 Setup

Using S3 with Glacier lifecycle option is great choice for backing up.

S3 の Glacier オプションは気楽に Glacier をつかえて便利です。

Editing

I choose machine to run Lightroom depending on number of photos. I use MacBook Air up to about 100 photos, but use Windows box for 100+ photos. Because MacBook Air is slow for editing.

MacBook Air と Windows box どちらかで Lightroom を使うんだけれど、それについては写真の枚数できめてる。枚数多いとちょっと処理速度はやくないとやってらんないけど、数十枚なら別に MacBook Air でいいや、という気。

And I import from locally on MacBook Air, but on Windows box, I import from local samba server; both connected with wired ethernet. I've not found any problems editing photos via network storages. I guess this doesn't make sense on wireless network.

Catalog is placed in local disk, because Lightroom doesn't support catalog on network storages. For securing catalog, Lightroom's catalog backup option is set to "each time when Lightroom exits."

MacBook Air は基本 Wi-Fi でつかっているからローカルにファイルおいていじってるけど、Windows box は有線でファイルサーバー (Linux + Samba) につながっているので SMB にファイルを置いて編集してる。wired network なのでそんな困らない。MacBook Air でこれやると破滅する。

After importing to Lightroom, I pick photos to publish or not by "P" and "X" and "U" shortcuts. Lightroom has flagging feature and I love it. As my experience, half of taken photos will be picked to publish.

Then finally, switch to Develop mode and edit picked photos. I don't take time for editing; adjust white balance, tune noise reduction, auto tone and adjust a bit, done.

Lightroom では P,X,U ショートカットでフラグをたてて写真を選定します。だいたい半分くらいになる。その後の現像もホワイトバランスいじって auto tone してちょっと修正する、みたいな事をやる程度でたいした事はしない。

Publishing

Publishing is just done in Lightroom's Flickr publishing integration, and local drive publishing. I upload published photos to local network storage and AWS S3.

Also, I'm using Flickr for some private photos (just for backing up or to show some of them to friends). Guest pass is nice feature to share privately.

Lightroom の Flickr とローカルの画像書き出し機能でだいたいすませてる。ローカルで書き出した画像は S3 とファイルサーバーに投げて保全してます。Flickr は Guest Pass でシェアするためとかバックアップとして、現像したけど公開しない写真も private 設定で一部あげています。

Published at 2014-01-18 20:23:13 +0900 | Permalink
2014-01-13

Looking back at 2013

Looking back at 2013.

2013 年が終わったのでここで振りかえっておこうと思う。

At the statistics

481 GitHub contributions

I've made 481 GitHub contributions (incl. private repositories: 1227) in 2013.

これはさっきとったキャプチャなので数値が変化しているけど、年末にみた Year of Contributions は 481 でした。プライベートリポジトリを含めると 1227。

主に個人で管理しているサーバーの puppet manifest 等が private repository として github.com に入っているのでその辺とか、あとはこっそりとまだ作っている物の中途半端な repository がカウント外。

22182 tweets

In 2013, I'd not tweeted so much at @sora_h account. There're 33756 tweets at other, my private accounts.

2013 年は @sora_h での発言は割とすくなかった気がします。他のアカウントの合算が 33756 とかなので、それでもあわせて 55938 tweets か。そんなもんですね。もうちょっと多い気がしたけど。

3450 pictures

Uploaded 3450 pictures on Flickr. I'm still using E-PL1 which released in 2010, but it seems to be outdated. Thinking about to get new camera in 2014. E-PL6?

And I bought fixed focus lens in May 2013. I've used it for most pictures in 2013: LUMIX G 20mm/F1.7 ASPH.

3450 枚の写真を Flickr にアップロードしたようです。そろそろ E-PL1 じゃ厳しい気がしてきた (ISO とか…) ので、E-PL6 あたりに買い替えたい。

RubyKaigi 2013 頃に単焦点レンズを買って、それ以降はほぼそれだけで過ごしてる気がする。わりと iPhone とかでもそうだけど、ズームいらない。カンファレンスの時くらい。

http://www.flickr.com/photos/sora_h/archives/date-taken/2013/calendar/

29 conferences/events

Attended 29 conferences/events/meet-ups. But, I had no talks in 2013. I should, and try to, make something to talk in 2014...

ISUCON (performance tuning contest in 8 hrs) was very interested event in 2013. I want to attend next time.

ざっとカレンダー見て数えたところ、29 個くらいカンファレンス・勉強会に参加したみたいです。 わりといろいろインプットはしたつもりだけど、逆に言えばあまりアウトプットがなく、実は今年は登壇してなかったりします。

なかでも ISUCON は 2013 年に参加したイベントの中でかなり盛り上がれたものでした。今年もあればまた参加したい。

2 gems

Released 2 (tiny) gems.

ちいさいもので以下 2 つをリリースしてました。

あまりたいしたことないですね。何か成果っぽい物を出したい。

28 books, 127 comics

http://booklog.jp/users/soraher/chart/2013

マンガばかり読んでた年だった。本も…だいたいラノベだな。何も言えない…

しかしブクログ見返してみるとその頃の疲弊具合がだいたいわかるので面白いなー。これは私本人にしかわかんないかな。

技術書についてはブクログで読了しわすれてるのがだいたいなのでカウントできないです。

177 amazon orders

Almost books and comics and CDs.

Also I checked my iTunes Library and it shows 1404 tracks (13.9 GB) added in 2013. About +1GB/mo? Looking forward for iPhone with 128GB storage.

いろいろ買ってますね。だいたい本と CD でした。

iTunes ライブラリを眺めたところ、2013 年は 1404 tracks, 13.9 GB の増加とのことでした。一月にだいたい 1GB 増えてるのか。iPhone の 128GB はやく出ないもんかなあ。

2 trips

There were 2 trips in 2013 (Sapporo and Gunma.) I'm planning to go US and Sapporo (again) in this year.

年始の社員旅行を含めると 2 箇所 (群馬、札幌) に遠出しました。今年は US とか行きたいです。あとまた札幌。

Activity

I've attended many events and conferences, but, I couldn't publish any big products/libraries in 2013. It doesn't mean I was contributing open source projects. I'll try to contribute or make something this year.

2013 年は割とイベントにでて交流はしていたけれど、成果としてはあまり何かを外に出す事ができていない年でした。いくつか時間を見つけては自分で使うための Sinatra アプリケーション等をつくったりしてたけど、まだ未完成だったり中途半端だった。

かといって、open source 活動ができていると言えばそれもできてない。小さいライブラリとかさえあまり出せてなかった。今年、2014 年はもうちょっと活動したいですね。2013 年は登壇とかできてないしね…

So, 2014

As I wrote above, my open source works in 2013 was too little bit. I try to make something in 2014... and I like to learn another language (Scala? Go? Haskell?)

btw, I guess I have to improve my English skill...

というわけで 2013 年は時間あったのに何もやれなかったなあという感じでした。今年はなにか出して発表できるといいなーと思ってます…

英語力もおわかりのように適当で中途半端なのでそれも精進したいなあというのと、そろそろ Go とか Scala とか Haskell に手をちゃんと出してみたいなあと思いつつ時間がとれてない。

がんばります…

Published at 2014-01-13 06:59:06 +0900 | Permalink
2013-07-28

render_to_string doesn't work well in ActionController::Live

render_to_string doesn't work well in ActionController::Live.

Because render_to_string modifies response_body and restores it. But #response_body= regenerates response.stream and ActionController::Live's overridden #response_body= closes response.stream, so response.stream.write won't work after any render_to_string in ActionController::Live.

def render_to_string(*)
  orig_stream = response.stream
  super
ensure
  if orig_stream
    response.instance_variable_set(:@stream, orig_stream)
  end
end

Above code works well as monkey patch to fix this issue.

I requested to pull this to upstream: https://github.com/rails/rails/pull/11623

Published at 2013-07-28 00:54:55 +0900 | Permalink
2013-07-04

Deny incoming packets via IPv6 except from link local address on OS X

ip6fw add 63500 allow tcp from any to any established
ip6fw add 63500 allow ipv6-icmp from any to any
ip6fw add 64000 deny ipv6 from not fe80::/64 to any in
ip6fw add 65000 allow ipv6 from fe80::/64 to any

See also: ip6fw(8)

Published at 2013-07-04 14:56:03 +0900 | Permalink
2013-01-29

Class Variables and Instance Variables on Class, in Ruby

Do you know problems around class variables in Ruby?

Class variable

You can declare class variables by using @@ for prefix of variable name, for instance: @@foo.

Problem

But, class variables can easily overwrite by subclasses. This is based on Ruby specification; class variables can be shared on its subclass.

Class variables are similar with global variables. They're too hard to handle safely.

For usually cases, I can't recommend to use.

Declare Instance Variable on Class object

So then, how we define "class variable," safely?

In Ruby, classes are object. This means you can define instance variable on class.

Scope of instance variables on class are closed within the same class. Thus, they don't effect on subclasses.

(Of course you can use attr_accessor, attr_reader, attr_writer. Example code)

Using from instance

Here are how to use that variables from instance objects.

Use attr_accessor

The simple solution.

Use instance_variable_get

but if you wanted to protect from foreigns, you can use instance_variable_get and private method.

Published at 2013-01-29 02:29:02 +0900 | Permalink
2013-01-17

Run rbenv in global

Updated (Feb 13, 2014): Fixed my poor English. And note that I hadn't encountered problem around my global rbenv installation for a year.

When we want to have ruby in a server, sometime we can't use the system's package manager.

Because package managers (in almost distros) serve older Ruby. Plus, they may have weird complex rubygems integration.

So the following are my ideas to get Ruby in your servers:

  • Build packages by ourselves: deb, rpm, ebuild

    • I think helpful when you have many servers, but when not, it might be overkill.
    • Building and maintaining packages are boring stuff.
  • Install ruby by hand: make && make install

    • Difficult to control later. What you do when you want to uninstall ruby?
    • How to upgrade cleanly?
  • Version Managers: Install rbenv, rvm

    • I think this is the bet solution; at least for me.
    • I prefer rbenv because it's made simple. rvm changes many shell behaviors.
    • rbenv is great for installing globally.

So, I'm using rbenv for my servers.

But Ruby is used from many users (including system users, daemons). Normally we install rbenv in ~/.rbenv, but it's depends on user's shell ($PATH) configuration and user's home directory.

Here's how I installed rbenv in global.

Install rbenv and ruby-build

I used /usr/local/rbenv for place rbenv.

sudo -i

cd /usr/local
git clone https://github.com/sstephenson/rbenv rbenv
mkdir -p rbenv/plugins
git clone https://github.com/sstephenson/ruby-build rbenv/plugins/ruby-build

RBENV_ROOT

rbenv looks RBENV_ROOT for many paths. In default (= when not specified), ~/.rbenv is set.

When installed rbenv in /usr/local/rbenv, we should set it to RBENV_ROOT=/usr/local/rbenv.

Install ruby in rbenv by ruby-build

Simply do:

sudo env RBENV_ROOT=/usr/local/rbenv install 1.9.3-p374
sudo env RBENV_ROOT=/usr/local/rbenv rehash

Then, set installed ruby to default ruby:

sudo env RBENV_ROOT=/usr/local/rbenv global 1.9.3-p374

Okay, we've done setup ruby on /usr/local/rbenv/shims/ruby. but, still requires to set $PATH. Let's solve it.

global-rehash

I wrote simple plugin for rbenv: https://github.com/sorah/rbenv-global-rehash

This generates symbolic links and scripts from ${RBENV_ROOT}/shims and ${RBENV_ROOT}/bin into specified directory.

For ${RBENV_ROOT}/bin, this plugin generates script to invoke with suitable $RBENV_ROOT.

But scripts in ${RBENV_ROOT}/shims (generated by rbenv rehash) already contains export RBENV_ROOT, so this script generates symlink for shims.

Install this:

sudo git clone https://github.com/sorah/rbenv-global-rehash /usr/local/rbenv/plugins/rbenv-global-rehash

Then,

sudo env RBENV_ROOT=/usr/local/rbenv rbenv global-rehash /usr/local/bin

Finally you can:

/usr/local/bin/rbenv versions
/usr/local/bin/ruby -v

# RBENV_ROOT will not be required after the first run
sudo rbenv global-rehash /usr/local/bin

Enjoy!

Published at 2013-01-17 22:56:13 +0900 | Permalink
2013-01-04

Renewed this blog

Just released days.gem, the simple blog system built up with Sinatra, and migrated this blog from Lokka.

And, I'll never write Japanese-only article for this blog, please see also http://diary.sorah.jp/ for Japanese entries.

Days

Days is simple blog system built up with Ruby + Sinatra.

This system's big feature is: core separated from deployment.

To set up days and start it, we have to do:

$ gem install days
$ mkdir my_blog
$ cd my_blog
$ days init
$ days migrate
$ days server

by days init, days.gem generates simple Gemfile, config.yml, and config.ru.

But you can customize appearances by placing haml views to views directory.

Core application part is included in days.gem, and it's required by config.ru and gem bundler.

I'm writing tutorial and documentation of days.gem. Stay tuned, please!

Published at 2013-01-04 05:27:50 +0900 | Permalink
2012-12-17

Show warning for RSpec examples that has no expectation

dev

You may sometimes forget to write should for matchers.

it { a == b } # ng
it { a.should == b } # ok

Using the above RSpec configuration, that warns for such cases.

$ rspec -fd spec/a_spec.rb
foo
[WARN] No expectation in example at ./spec/a_spec.rb:4: You may forget to write `should` in the example
  example at ./spec/a_spec.rb:4
Published at 2012-12-17 10:33:05 +0900 | Permalink
2012-11-30

Updating Galaxy Nexus (JP) from stock 4.0 to CM10 with SIM unlocking

Finally got latest Android world.

Before

  • Device: Galaxy Nexus (JP, locked to docomo)
  • Android 4.0.0
  • SIM: SoftBank

    • Unlocked Using F*ckDocomo

Update this to CyanogenMod 10 (Android 4.2.)

Lock again

"Unlock → Lock again." on F*ckDocomo.apk.

Backup /factory/nv_data.bin.

Backup

$ adb backup -f backup.ab -apk -all

Then enter your password on the device.

$ fastboot boot recovery-clockwork-…

Get latest clockworkmod and boot it, then run nandroid backup.

Update

Apply CM10 and gapps zip.

then, reboot and edit backup'd nv_data.bin by seeing this entry.

$ adb push nv_data.bin /sdcard/nv_data.bin # push edited nv_data.bin
$ adb shell
$ su
# mount -o remount,rw -t ext4 /dev/block/mtdblock0 /factory
# cd /factory
# cp /sdcard/nv_data.bin .
# chown radio.radio nv_data.bin
# chmod 700 nv_data.bin
# cp nv_data.bin /data/radio/
# reboot

after reboot, your device is still locked, so:

$ adb shell
$ su
# cat /data/radio/log/nv.log
...
Fri Nov 30 00:44:45 2012: MD5 fail. orignal md5 'aaa' computed md5 'bbb' (rild)
Fri Nov 30 00:44:45 2012: Backup NV restored.(GED)
# mount -o remount,rw -t ext4 /dev/block/mtdblock0 /factory
# cd /factory
# echo -n 'bbb' > nv_data.bin.md5
# chown radio.radio nv_data.bin.md5
# chmod 700 nv_data.bin.md5
# cp nv_data.bin.md5 /data/radio/
# reboot

finally your device is unlocked.

Restore

$ adb restore backup.ab
Published at 2012-11-30 01:11:28 +0900 | Permalink
2012-10-13

rspec-mock broken by `scope :public` in ActiveRecord::Base

dev
class Foo < ActiveRecord::Base
  scope :public, -> { … } # this overrides Module#public
end

class FoosController < ApplicationController
  def create
    @foo = Foo.new(params[:foo])

    respond_to do |format|
      if @foo.save
        format.html { redirect_to @foo, notice: 'foo was successfully created.' }
        format.json { render json: @foo, status: :created, location: @foo }
      else
        format.html { render action: "new" }
        format.json { render json: @foo.errors, status: :unprocessable_entity }
      end
    end
  end
end

describe FoosController do
  describe "POST create" do
    describe "with invalid params" do
      it "re-renders the 'new' template" do
        # rspec-mock expects Foo.public to `Module#public`
        Foo.any_instance.stub(:save).and_return(false)

        post :create, {:foo => {}}, valid_session
        response.should render_template("new")
      end
    end
end

scope :public overrides Module#public method in the model, but rspec-mock expects Foo.public to be Module#public. rspec-mock calls Foo.public with arguments, so it raises ArgumentError on line scope :public. :/

 ArgumentError:
   wrong number of arguments (1 for 0)

こんな感じで scope :public, -> {…} していると、rspec-mock が Module#public(*name) を期待して引数付きで public を呼んで崩壊する模様。

Foo を stub ってるせいでテストがコケるみたいです。

Published at 2012-10-13 01:06:17 +0900 | Permalink
2012-10-08

Pending は邪悪な物であり、邪悪な物を生み出した人間には斧を

そらはーです!

RSpec で pending 使ってテストを一時的に無効化した事ある方は大勢いらっしゃるんじゃないでしょうか!

なんらかの事情で一時的に pending せざるを得ない状況ならともかく、pending したなら責任をもって該当のテストを治すか、そもそも不要なら消すなどといった対処をしてもらいたいものですね!

でも、実際来週までには!とか言っても放置する人間や、そもそも直さず1年,2年以上放置される事もしばしばあるのが現実です………

pending を放置する事によって、実はそれは(他の人にとって)かなり重要なテストで、そんなテストがpendingされてる訳ないと思った、他のメンバーによる変更で実はそのテストがコケて事故っていたという可能性も存在するわけです。

  1. 重要なテストをAさんが pending する (「テストは追って修正する」みたいな感じで)
  2. 比較的大きめの変更を B さんが加える。一応全部テストを回してグリーンな事を確認
  3. (実は重要なテストがAさんによって pending されている事に気づいていない)
  4. 事故っている

本当は pending を使わないのが一番なんですが、やむなく使った場合はちゃんと責任を持って直しましょ… 直せ!!! というわけで、催促するツールを作りました!

Pendaxes

cookpad/pendaxes

pend(ing) axes です! 今のところ rspec をサポートしてます!

メール等でコミットした人に個別で催促メールを送ったりできます!!!!

$ gem install pendaxes
$ cd /path/to/your/git/working/copy
$ pendaxes --init
$ pendaxes
$ open report.html

とりあえず試すには、working copy の中で pendaxes --init を叩くとそのディレクトリで .pendaxes.yml を吐き出すので、そこで pendaxes を実行すると report.html を吐き出します。

後述するメール設定をすると、この report.html のようなメールがコミッター毎にその人のコミットしたpendingがメールで送られると思えば良いです。

(git grep とかを使ってるので git リポジトリと git が必要です)

(Ruby 1.9.2 以降で動きます。それ以前は保証しません)

設定ファイル

https://github.com/cookpad/pendaxes/wiki/Configuration

を参照してください… ちょっと複雑です。以下に一例を載せるので参考にしてください><

メールを送る

notifications に以下のような行を足せば動きます。mail.gemsendmail delivery_method を使って、各コミッターにメールを送信します。

- use: mail
  reporter:
    use: haml
  from: no-reply@example.com
  delivery_method: sendmail

こんなメールが飛びます:

cron で回してみる

cron で回す際は、config.yml に git clone 先と clone 元を指定してあげると pendaxes が pull まで面倒を見てくれて便利です。pendaxes --init で指定付きで生成できます。

$ pendaxes --init config.yml https://github.com/cookpad/pendaxes.git /tmp/pendaxes

で、/tmp/pendaxeshttps://github.com/cookpad/pendaxes.git を clone して自動で git fetchgit reset --hard で最新のコードに更新してくれる config.yml を生成してくれます

$ pendaxes config.yml

で生成した config.yml を実行してくれます。後は同じように config.yml を編集してメール等を送る設定を追加した上で cron で回せばいいと思います。

効果

毎朝6時に cron で回して送るようにしたところ、だいたい放置されていた大量の pending がどんどん数日で消えて行きました! わーい!

…あれ?

Published at 2012-10-08 18:10:37 +0900 | Permalink