2015-12-16

Hello HBase

Meet HBase Again, also Phoenix.

Install HBase

A.配置213到其他节点的免密码登陆, 213作为Master节点.

ssh-keygen
scp id_rsa.pub 192.168.47.241:~/
ssh 192.168.47.241

cd .ssh 
cat ~/id_rsa.pub >> authorized_keys

要登录到目标节点添加授权key. 一种更简单的做法是ssh-copy-id host

B.在213上修改hbase如下配置文件:

1) hbase-env.sh:

1 2	export JAVA_HOME=/usr/java/jdk1.7.0_51 export HBASE_MANAGES_ZK=false

2) hbase-site.xml:

<configuration>
    <property>
     <name>hbase.rootdir</name>
     <value>hdfs://tdhdfs/hbase</value>
    </property>
    <property>
     <name>hbase.cluster.distributed</name>
     <value>true</value>
    </property>
    <property>
    <name>hbase.master</name>
    <value>192.168.47.213:60000</value>
    </property>
    <property>
     <name>hbase.zookeeper.quorum</name>
     <value>192.168.47.83,192.168.47.84,192.168.47.86</value>
    </property>
    <property>
      <name>hbase.zookeeper.property.dataDir</name>
      <value>/home/qihuang.zheng/data</value> 
    </property>
</configuration>

因为213作为master, 所以指定hbase.master. 并没有专门的配置文件来指定master的地址.

3) 设置集群的所有节点 regionservers:

1
2
3

192.168.47.241
192.168.47.242
...

4) 拷贝HADOOP的hdfs-site.xml和core-site.xml到conf下(因为HDFS是HA)

C.分发hbase目录到集群中所有节点

1 2	scp -qr hbase-1.0.2 192.168.47.241:~/ ...

D.启动HBase集群: bin/start-hbase.sh

单独启动HMaster: bin/hbase-daemon.sh start master
单独启动所有的RegionServer: bin/hbase-daemons.sh start regionserver
单独启动一个RegionServer: bin/hbase-daemon.sh start regionserver

修改conf/hbase-env.sh的pid文件夹路径,默认是在/tmp下

查看进程：

[admin@spark047213 logs]$ jps -lm | grep hbase
28722 org.apache.hadoop.hbase.regionserver.HRegionServer start
28535 org.apache.hadoop.hbase.master.HMaster start
HQuorumPeer    #单个节点下，没有这个进程

E.测试HBase的CRUD: hbase shell

status
list
create 'table','cf'
put 'table','rowkey1','cf:column','value'
get 'table','rowkey1'
put 'table','rowkey1','cf:','value'
scan 'table'

F.查看HBase进程. HMaster和HRegionServer

G.访问: http://localhost:16010

1	map.exp 213 16010

hbase_ui

Install Phoenix

拷贝phoenix-4.6.0-HBase-1.0-server.jar或者所有phoenix相关jar包到HBase集群中每个节点

1 2	cp ~/phoenix-4.6.0-HBase-1.0-bin/phoenix.jar ~/hbase-1.0.2/lib/ scp ~/phoenix-4.6.0-HBase-1.0-bin/phoenix.jar 192.168.47.241:~/hbase-1.0.2/lib/

重启HBase集群的所有RegionServer节点: (这里由于是测试,直接重启HBase集群)

1 2	stop-hbase.sh start-hbase.sh

如果发现一直停不了,可以先hbase-daemon.sh stop master,再执行stop-hbase.sh关闭其他节点的RegionServer

[qihuang.zheng@spark047213 bin]$ ./sqlline.py
Zookeeper not specified.
Usage: sqlline.py <zookeeper> <optional_sql_file>
Example:
 1. sqlline.py localhost:2181:/hbase
 2. sqlline.py localhost:2181:/hbase ../examples/stock_symbol.sql

[qihuang.zheng@spark047213 bin]$ ./sqlline.py 192.168.47.83,192.168.47.84,192.168.47.86:2181:/hbase
Setting property: [isolation, TRANSACTION_READ_COMMITTED]
issuing: !connect jdbc:phoenix:192.168.47.83,192.168.47.84,192.168.47.86:2181:/hbase none none org.apache.phoenix.jdbc.PhoenixDriver
Connecting to jdbc:phoenix:192.168.47.83,192.168.47.84,192.168.47.86:2181:/hbase
15/12/16 20:45:25 INFO metrics.Metrics: Initializing metrics system: phoenix
15/12/16 20:45:29 INFO client.ConnectionManager$HConnectionImplementation: Closing master protocol: MasterService
15/12/16 20:45:29 INFO client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x1515d5299ea2892
Connected to: Phoenix (version 4.6)
Driver: PhoenixEmbeddedDriver (version 4.6)
Autocommit status: true
Transaction isolation: TRANSACTION_READ_COMMITTED
Building list of tables and columns for tab-completion (set fastconnect to true to skip)...
88/88 (100%) Done
Done
sqlline version 1.1.8
0: jdbc:phoenix:192.168.47.83,192.168.47.84,1>

实际上sqlline.py后面跟的地址可以只有一个ZooKeeper的地址,可以不需要端口号和后面的/hbase

如果启动sqlline.py进不了>提示符, 可以修改log级别为DEBUG,查看在哪里出错了.

1
2
3

15/12/16 18:03:17 DEBUG util.Shell: Failed to detect a valid hadoop home directory
java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
    at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:302)

给每个节点的.bashrc加上HADOOP_HOME环境变量,并重启HBase集群

CRUD

0: jdbc:phoenix:192.168.47.83,192.168.47.84,1> !tables
+------------------------------------------+------------------------------------------+------------------------------------------+------------------------------------------+------------------------+
|                TABLE_CAT                 |               TABLE_SCHEM                |                TABLE_NAME                |                TABLE_TYPE                |                 REMARK |
+------------------------------------------+------------------------------------------+------------------------------------------+------------------------------------------+------------------------+
|                                          | SYSTEM                                   | CATALOG                                  | SYSTEM TABLE                             |                        |
|                                          | SYSTEM                                   | FUNCTION                                 | SYSTEM TABLE                             |                        |
|                                          | SYSTEM                                   | SEQUENCE                                 | SYSTEM TABLE                             |                        |
|                                          | SYSTEM                                   | STATS                                    | SYSTEM TABLE                             |                        |
+------------------------------------------+------------------------------------------+------------------------------------------+------------------------------------------+------------------------+

貌似之前启动HBase之后用hbase shell测试的table并没有在这里被查询到.

create table IF NOT EXISTS test.Person (IDCardNum INTEGER not null primary key, Name varchar(20),Age INTEGER);
upsert into test.Person (IDCardNum,Name,Age) values (100,'小明',12);
upsert into test.Person (IDCardNum,Name,Age) values (101,'小红',15);
upsert into test.Person (IDCardNum,Name,Age) values (103,'小王',22);
select * from test.Person;

shell创建的表在Phoenix中看不见

在hbase shell中手动指定表的前缀,相当于数据库名: test.table

hbase(main):008:0* create 'test.table','cf'
0 row(s) in 0.9420 seconds

=> Hbase::Table - test.table
hbase(main):009:0> list
TABLE
SYSTEM.CATALOG
SYSTEM.FUNCTION
SYSTEM.SEQUENCE
SYSTEM.STATS
TEST.PERSON
table
test.table
7 row(s) in 0.0200 seconds

=> ["SYSTEM.CATALOG", "SYSTEM.FUNCTION", "SYSTEM.SEQUENCE", "SYSTEM.STATS", "TEST.PERSON", "table", "test.table"]
hbase(main):010:0>

在shell中能看的之前在phoenix中创建的test.person, 但是在phoenix中并没有看到在shell中创建的test.table:

0: jdbc:phoenix:192.168.47.83,192.168.47.84,1> !table
+------------------------------------------+------------------------------------------+------------------------------------------+------------------------------------------+------------------------+
|                TABLE_CAT                 |               TABLE_SCHEM                |                TABLE_NAME                |                TABLE_TYPE                |                 REMARK |
+------------------------------------------+------------------------------------------+------------------------------------------+------------------------------------------+------------------------+
|                                          | SYSTEM                                   | CATALOG                                  | SYSTEM TABLE                             |                        |
|                                          | SYSTEM                                   | FUNCTION                                 | SYSTEM TABLE                             |                        |
|                                          | SYSTEM                                   | SEQUENCE                                 | SYSTEM TABLE                             |                        |
|                                          | SYSTEM                                   | STATS                                    | SYSTEM TABLE                             |                        |
|                                          | TEST                                     | PERSON                                   | TABLE                                    |                        |
+------------------------------------------+------------------------------------------+------------------------------------------+------------------------------------------+------------------------+

0: jdbc:phoenix:192.168.47.83,192.168.47.84,1> select * from test.table;
Error: ERROR 604 (42P00): Syntax error. Mismatched input. Expecting "NAME", got "table" at line 1, column 20. (state=42P00,code=604)
org.apache.phoenix.exception.PhoenixParserException: ERROR 604 (42P00): Syntax error. Mismatched input. Expecting "NAME", got "table" at line 1, column 20.
    at org.apache.phoenix.exception.PhoenixParserException.newException(PhoenixParserException.java:33)
    at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:111)
    at org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1285)
    at org.apache.phoenix.jdbc.PhoenixStatement.parseStatement(PhoenixStatement.java:1366)
    at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1429)
Caused by: MismatchedTokenException(140!=98)
    at org.apache.phoenix.parse.PhoenixSQLParser.recoverFromMismatchedToken(PhoenixSQLParser.java:352)
0: jdbc:phoenix:192.168.47.83,192.168.47.84,1> select * from test.person;
+------------------------------------------+----------------------+------------------------------------------+
|                IDCARDNUM                 |         NAME         |                   AGE                    |
+------------------------------------------+----------------------+------------------------------------------+
| 100                                      | 小明                   | 12                                       |
| 101                                      | 小红                   | 15                                       |
| 103                                      | 小王                   | 22                                       |
+------------------------------------------+----------------------+------------------------------------------+
3 rows selected (0.129 seconds)

实际上可以在Phoenix中创建视图来将hbase中的表映射进来: http://phoenix.apache.org/faq.html#How_I_map_Phoenix_table_to_an_existing_HBase_table

其他

删除ZK中的hbase节点

1
2
3

zkCli.sh -server zkHost1,zkHost2
rmr /hbaseSplitWAL
rmr /hbase

Snapshot

http://wangneng-168.iteye.com/blog/2154669

CopyTable的功能可以在同一个集群复制不同的表，或者不同集群复制相同的表。
不用CopyTable，因为CopyTable工具采用scan查询，写入新表时采用put和delete API，全是基于hbase的client api进行读写，无法使用Import工具的bulk导入

HBase提供快照功能，下面三个命令是在一个集群内的快照维护/管理。

对表做快照：snapshot ‘表名’ ‘快照名’
将快照克隆一份新的表：clone_snapshot ‘快照名’ ‘新的表名’
恢复快照：disable ‘表名’；restore_snapshot ‘快照名’

克隆快照指定是将快照克隆出一份新的表。在对表做快照后，这份快照的数据结构已经确定了，克隆后的是一张新的表。新表的内容和快照的内容完全一样。
恢复快照需要先把表禁用，恢复快照时，不需要指定表名。因为快照中已经有表名。恢复快照类似于数据库的恢复、回滚技术。

限制条件：

clone_snapshot快照时，表名必须不能存在。不允许clone成一个已经存在的表。
restore_snapshot必须禁用表。如果表已经存在，要从快照恢复数据，只能用restore，不能用clone

将源集群(srv1)的表拷贝到目标集群(srv2)分成两个步骤：

在源集群执行以下两个命令：

$ snapshotTable="MyTable"
$ snapshotName="snapshot-1"

$ bin/hbase shell -e "snapshot '$snapshotTable', '$snapshotName'"
$ bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot \
  -snapshot $snapshotName -mappers 16 -bandwidth 10 \
  -copy-to hdfs://srv2:8082/hbase

hdfs://srv2:8082/hbase是目标hbase集群的root.dir

在目标集群执行：

$ snapshotTable="MyTable"
$ snapshotName="snapshot-1"

$ bin/hbase shell -e "disable '$snapshotTable';restore_snapshot '$snapshotName';enable '$snapshotTable'"

第一步的ExportSnapshot只是把快照数据同步到目标集群，还需要恢复才可以将快照运用到目标集群。
但是恢复操作需要禁用表，对于一直在线的应用而言，不可接受。

In case of partial data-loss due to misbehaving client,
instead of a full restore that requires the table to be disabled,
you can clone the table from the snapshot and
use a Map-Reduce job to copy the data that you need, from the clone to the main one.

因为完全恢复（full restore）需要禁用表时，写入到目标集群表中的数据会丢失。
不过你可以使用clone_snapshot复制出一张表（新表），然后使用MR作业将新表复制到旧表，

一些问题：

clone_snapshot能不能直接复制旧表。比如两个tableName都一样，是否可行？

snapshot ‘tableName’ ‘快照名’
clone_snapshot ‘快照名’ ‘tableName’

hbase(main):007:0> snapshot 'table_1','snapshot_10'
hbase(main):007:0> clone_snapshot 'snapshot_10','table_1'

ERROR: Table already exists: table_1!

开启replciation时，snapshot会有影响吗？

Since Replication works at log level and snapshots at file-system level,
after a restore, the replicas will be in a different state from the master.
If you want to use restore, you need to stop replication and redo the bootstrap.

如果该表开启了replication功能实现主从同步则在使用restore_snapshot功能对主表进行恢复时，
必须先停止replication功能并且redo the bootstrap，
因为replication是基于WAL日志实现的，而snapshot是直接在hdfs层面不是基于hbase的put、delete等API进行的，不会写WAL日志。

什么是redo bootstrap?重启节点吗？

HBase版本号的一个坑

本文标题:Hello HBase

文章作者:任何忧伤,都抵不过世界的美丽

发布时间:2015年12月16日 - 00时00分

最后更新:2019年02月14日 - 21时42分

原始链接:http://github.com/zqhxuyuan/2015/12/16/2015-12-16-Apache-HBase/

许可协议: "署名-非商用-相同方式共享 3.0" 转载请保留原文链接及作者。

↑
招人广告：对蚂蚁金服中间件感兴趣的可以发邮件到：qihuang.zqh at antfin.com