Raspberry Pi 3に Selenium Webdriverを入れて Webを自動で制御する方法

$ uname -a
Linux raspberrypi 4.9.41-v7+ #1023 SMP Tue Aug 8 16:00:15 BST 2017 armv7l GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 9.1 (stretch)
Release:        9.1
Codename:       stretch

# お決まりの手順
sudo apt-get update

# 必要なバイナリ類をインストールする
# Iceweaselは Debianでの Firefoxの商標対策の名称
sudo apt-get -y install iceweasel

iceweasel -V
# Mozilla Firefox 52.4.0

firefox-esr -V
# Mozilla Firefox 52.4.0

# Debianでの Firefoxの商標問題が解決したので firefox-esr
# （firefox-esrは上の iceweaselでインストールされます）
sudo apt-get -y install firefox-esr

# Xvfb 仮想ディスプレイのソフトウェア X virtual framebuffer
sudo apt-get -y install xvfb

# Selenium WebDriverで必要
sudo apt-get -y install ruby-dev

# Selenium WebDriverをインストールする
# 最新の selenium-webdriver (3.6.0)は geckodriverを必要とし、その geckodriverは実質使い物にならないので古いバージョンを指定してインストールする
# --no-documentでドキュメントをインストールしない（時間短縮）
sudo gem install selenium-webdriver -v 2.53.4 --no-document

# headlessをインストールする
sudo gem install headless --no-document

pi@raspberrypi:~ $ gem list

*** LOCAL GEMS ***

bigdecimal (1.2.8)
childprocess (0.8.0)
did_you_mean (1.0.0)
ffi (1.9.18)
headless (2.3.1) ★★★
io-console (0.4.5)
json (1.8.3)
minitest (5.9.0)
net-telnet (0.1.1)
power_assert (0.2.7)
psych (2.1.0)
rake (10.5.0)
rdoc (4.2.1)
rubyzip (1.2.1)
selenium-webdriver (2.53.4) ★★★
test-unit (3.1.7)
websocket (1.2.4)

#!/usr/bin/env ruby

require 'selenium-webdriver'
require 'headless'

headless = Headless.new
headless.start

driver = Selenium::WebDriver.for :firefox
driver.navigate.to 'https://www.google.com/'

puts driver.title
driver.save_screenshot('google_1.png')
open('google_1.html', 'w'){|f| f.write driver.page_source }

driver.find_element(:name, 'q').send_keys('猫 写真')
# driver.find_element(:name, 'btnK').click
# driver.find_element(:link_text => 'Google 検索').click
driver.find_element(:name, 'btnG').click
# driver.find_element(:link_text => '検索').click

puts driver.title
driver.save_screenshot('google_2.png')

driver.quit
headless.destroy

Google
猫 写真 - Google 検索

sudo gem uninstall selenium-webdriver
sudo gem install selenium-webdriver --no-document
# Successfully installed selenium-webdriver-3.6.0

/var/lib/gems/2.3.0/gems/selenium-webdriver-3.6.0/lib/selenium/webdriver/common/service.rb:59:in `binary_path':  Unable to find Mozilla geckodriver. Please download the server from https://github.com/mozilla/geckodriver/releases and place it somewhere on your PATH. More info at https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette/WebDriver. (Selenium::WebDriver::Error::WebDriverError)
        from /var/lib/gems/2.3.0/gems/selenium-webdriver-3.6.0/lib/selenium/webdriver/common/service.rb:49:in `initialize'

v3.0.0-beta1
============

IMPORTANT CHANGES

* Support for Firefox is via Mozilla's geckodriver. You may download
  this from https://github.com/mozilla/geckodriver/releases

wget https://github.com/mozilla/geckodriver/releases/download/v0.19.0/geckodriver-v0.19.0-arm7hf.tar.gz
tar -xzf geckodriver-v0.19.0-arm7hf.tar.gz

ls -l geckodriver
# -rwxr-xr-x 1 pi pi 6421011 Sep 16 10:57 geckodriver
# chmod +x geckodriver

# geckodriverを /usr/local/binに移動する
sudo mv geckodriver /usr/local/bin

$ geckodriver -V
geckodriver 0.19.0

The source code of this program is available from
testing/geckodriver in https://hg.mozilla.org/mozilla-central.

This program is subject to the terms of the Mozilla Public License 2.0.
You can obtain a copy of the license at https://mozilla.org/MPL/2.0/.

/usr/lib/ruby/2.3.0/net/protocol.rb:158:in `rbuf_fill': Net::ReadTimeout (Net::ReadTimeout)
        from /usr/lib/ruby/2.3.0/net/protocol.rb:136:in `readuntil'
        from /usr/lib/ruby/2.3.0/net/protocol.rb:146:in `readline'

client = Selenium::WebDriver::Remote::Http::Default.new
client.timeout = 600

# WARN Selenium [DEPRECATION] :timeout= is deprecated. Use #read_timeout= and #open_timeout= instead.

client = Selenium::WebDriver::Remote::Http::Default.new
client.open_timeout = 600
client.read_timeout = 600

driver = Selenium::WebDriver.for :firefox, :http_client => client
driver.navigate.to 'https://www.google.com/'

stack backtrace:: connection refused (Selenium::WebDriver::Error::UnknownError)
        from    0:   0x51a747 - backtrace::backtrace::trace::h736111741fa0878e
        from    1:   0x51a8af - backtrace::capture::Backtrace::new::h63b8a5c0787510c9
        from    2:   0x46e023 - webdriver::error::WebDriverError::new::hea6d4dbf778b2b24

sudo gem install chromedriver-helper --no-document
# Successfully installed chromedriver-helper-1.1.0

wget http://ftp.jp.debian.org/debian/pool/main/c/chromium-browser/chromium-driver_61.0.3163.100-1~deb9u1_armhf.deb

sudo dpkg -i chromium-driver_61.0.3163.100-1~deb9u1_armhf.deb

dpkg: dependency problems prevent configuration of chromium-driver:
 chromium-driver depends on libminizip1 (>= 1.1); however:

sudo rm -f /etc/apt/sources.list.d/google-chrome.list
sudo apt --fix-broken install

sudo apt-get install libminizip1

sudo dpkg -i chromium-driver_61.0.3163.100-1~deb9u1_armhf.deb

 chromium-driver depends on chromium (= 61.0.3163.100-1~deb9u1); however:
  Package chromium is not installed.

sudo apt-get install -f

pi@raspberrypi:~ $ sudo apt-get install chromium
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package chromium is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
  chromium-bsu

E: Package 'chromium' has no installation candidate

pi@raspberrypi:~ $ sudo apt-get install chromium-bsu

--
The following packages have unmet dependencies:
 chromium-bsu : Depends: chromium-bsu-data (>= 0.9.14) but it is not going to be installed
                Depends: fonts-uralic but it is not going to be installed or
                         ttf-uralic but it is not installable
                Depends: libalut0 (>= 1.0.1) but it is not going to be installed
                Depends: libglc0 (>= 0.7.1) but it is not going to be installed
                Depends: libsdl2-2.0-0 (>= 2.0.4) but it is not going to be installed
                Depends: libsdl2-image-2.0-0 (>= 2.0.1) but it is not going to be installed

sudo apt-get -y install libsdl2-image-2.0-0 libsdl2-2.0-0 libglc0 libalut0
sudo apt-get -y install fonts-uralic
# ttf-uralic
# E: Package 'ttf-uralic' has no installation candidate

sudo apt-get -y install chromium-bsu-data


sudo apt-get install chromium-bsu

sudo dpkg -i chromium-driver_61.0.3163.100-1~deb9u1_armhf.deb

npm install phantomjs

# PhantomJS not found on PATH
# Unexpected platform or architecture: linux/arm
# It seems there is no binary available for your platform/architecture
# Try to install PhantomJS globally

sudo npm install -g phantomjs

# npm WARN deprecated phantomjs@2.1.7
PhantomJS not found on PATH
Unexpected platform or architecture: linux/arm
It seems there is no binary available for your platform/architecture
Try to install PhantomJS globally

cd
git clone https://github.com/piksel/phantomjs-raspberrypi.git
chmod +x ~/phantomjs-raspberrypi/bin/phantomjs
sudo ln -s /home/pi/phantomjs-raspberrypi/bin/phantomjs /usr/bin/
phantomjs --version

sudo apt-get install libssl1.0.0 libssl-dev

pi@raspberrypi:~/phantomjs-raspberrypi/bin $ ./phantomjs --version
./phantomjs: error while loading shared libraries: libssl.so.1.0.0: cannot open shared object file: No such file or directory

Raspberry Pi Raspbian Jessieで PhantomJS 2.1.1をビルドする方法、OpenSSL 1.0.1t

Raspberry Pi Raspbian Stretchで PhantomJS 2.1.1をビルドする方法、OpenSSL 1.1の罠

Raspbianで最新版の Phantom.js 2.1.1のビルド済みバイナリを導入して速攻で簡単に動かす方法

sudo apt-get update
sudo apt-get -y install ruby
ruby -v
# ruby 2.3.3p222 (2016-11-21) [arm-linux-gnueabihf]

sudo apt-get -y install build-essential patch ruby-dev zlib1g-dev liblzma-dev

sudo gem install nokogiri --no-document
nokogiri -v
# Nokogiri (1.8.1)

require 'open-uri'
require 'nokogiri'

html = Nokogiri::HTML(open("http://www.yodobashi.com/product/100000001001887549/"))

itemName = html.css('h1#products_maintitle').first
itemNameText = itemName.inner_html
itemNameText.gsub!(/<br>/," ")
itemNameText.gsub!(/<span itemprop=\"name\">/,"")
itemNameText.gsub!(/<\/span>/,"")
puts itemNameText
# エレコム ELECOM TK-FDM063BK [無線2.4GHzフルキーボード/109キー]

price = html.css('span#js_scl_unitPrice').first
priceText = price.content
puts priceText
# ￥1,760
priceText.gsub!(/￥/,"")
priceText.gsub!(/,/,"")
puts priceText.to_i
# 1760

エレコム ELECOM TK-FDM063BK [無線2.4GHzフルキーボード/109キー]
￥1,760
1760

sudo apt-get update
sudo apt-get install exagear-desktop

sudo apt-get -y install qemu

sudo apt-get -y install git libglib2.0-dev libfdt-dev libpixman-1-dev zlib1g-dev
sudo apt-get -y install libnfs-dev libiscsi-dev

cd
mkdir qemu
cd qemu
# git clone git://git.qemu-project.org/qemu.git
sudo wget https://github.com/qemu/qemu/archive/v2.10.1.tar.gz
sudo tar -xvf v2.10.1.tar.gz

cd qemu-2.10.1

./configure
# mkdir: cannot create directory ‘config-temp’: Permission denied
# ERROR: failed to create temporary directory

sudo ./configure --target-list="i386-softmmu" --enable-sdl --prefix=/usr
# ERROR: User requested feature sdl
#        configure was not able to find it.
#        Install SDL2-devel

sudo apt-get -y install libsdl2-dev

sudo ./configure --target-list="i386-softmmu" --enable-sdl --prefix=/usr

make

# Now let's start a simple test:
qemu-system-x86_64 -L pc-bios

pi@raspberrypi:~/qemu/qemu-2.10.1 $ sudo ./configure --target-list="i386-softmmu" --enable-sdl --prefix=/usr
Install prefix    /usr
BIOS directory    /usr/share/qemu
binary directory  /usr/bin
library directory /usr/lib
module directory  /usr/lib/qemu
libexec directory /usr/libexec
include directory /usr/include
config directory  /usr/etc
local state directory   /usr/var
Manual directory  /usr/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path       /home/pi/qemu/qemu-2.10.1
C compiler        cc
Host C compiler   cc
C++ compiler      c++
Objective-C compiler cc
ARFLAGS           rv
CFLAGS            -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g
QEMU_CFLAGS       -I/usr/include/pixman-1  -pthread -I/usr/include/glib-2.0 -I/usr/lib/arm-linux-gnueabihf/glib-2.0/include  -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv  -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -I/usr/include/libpng16
LDFLAGS           -Wl,--warn-common -g
make              make
install           install
python            python -B
smbd              /usr/sbin/smbd
module support    no
host CPU          arm
host big endian   no
target list       i386-softmmu
gprof enabled     no
sparse enabled    no
strip binaries    yes
profiler          no
static build      no
pixman            system
SDL support       yes (2.0.5)
GTK support       no
GTK GL support    no
VTE support       no
TLS priority      NORMAL
GNUTLS support    no
GNUTLS rnd        no
libgcrypt         no
libgcrypt kdf     no
nettle            no
nettle kdf        no
libtasn1          no
curses support    no
virgl support     no
curl support      no
mingw32 support   no
Audio drivers     oss
Block whitelist (rw)
Block whitelist (ro)
VirtFS support    no
VNC support       yes
VNC SASL support  no
VNC JPEG support  no
VNC PNG support   yes
xen support       no
brlapi support    no
bluez  support    no
Documentation     no
PIE               no
vde support       no
netmap support    no
Linux AIO support no
ATTR/XATTR support yes
Install blobs     yes
KVM support       yes
HAX support       no
TCG support       yes
TCG debug enabled no
TCG interpreter   no
RDMA support      no
fdt support       yes
preadv support    yes
fdatasync         yes
madvise           yes
posix_madvise     yes
libcap-ng support no
vhost-net support yes
vhost-scsi support yes
vhost-vsock support yes
vhost-user support yes
Trace backends    log
spice support     no
rbd support       no
xfsctl support    no
smartcard support no
libusb            no
usb net redir     no
OpenGL support    no
OpenGL dmabufs    no
libiscsi support  yes
libnfs support    yes
build guest agent yes
QGA VSS support   no
QGA w32 disk info no
QGA MSI support   no
seccomp support   no
coroutine backend ucontext
coroutine pool    yes
debug stack usage no
crypto afalg      no
GlusterFS support no
gcov              gcov
gcov enabled      no
TPM support       yes
libssh2 support   no
TPM passthrough   no
QOM debugging     yes
Live block migration yes
lzo support       no
snappy support    no
bzip2 support     no
NUMA host support no
tcmalloc support  no
jemalloc support  no
avx2 optimization no
replication support yes
VxHS block device no

FWinSdCardImager SDカード イメージ書き込みアプリ、ラズパイの Raspbian OS、Jetson Nanoの Ubuntuの書き込みに便利

FWinPiFinder ラズベリーパイ IPアドレス発見アプリ。ARPコマンドでラズパイの IPアドレスを探索発見する

Raspberry Pi 3系のトラブルであるある第一位の電源トラブル、低電圧警報に関する情報のまとめ

Raspberry Piで CPUの脆弱性 Spectreと Meltdownの脆弱性をチェックする方法

Raspberry Pi Zero Wを海外通販の Pimoroni等での購入方法、購入できる通販ショップ一覧まとめ

Raspberry Pi 3で安定して使える相性の無い最適な microSDカードの種類のまとめ

Raspberry Pi 3 Model Bに専用カメラモジュール RaspiCamを接続する方法

Raspberry Pi 3の Linuxコンソール上で使用する各種コマンドまとめ

Raspberry Pi 3公式フォーラムの FAQの内容の日本語訳

Raspberry Pi 3で GPIO端子の I2C機能を有効化する方法

大人気の CPUボード、Raspberry Pi 3 Model Bで作ってみよう

Espressif ESP8266 Arduino互換でスケッチが使える ESP-12Eモジュール基板

BangGood通販はドローン以外にも面白い商品がまだまだ有った（電子工作編）