Cassandra源码分析

cassandra

开坑：Cassandra-2.1源码分析

IDEA中运行Cassandra-2.1

1
2
3
➜  cassandra git:(cassandra-2.1) ✗ pwd
/Users/zhengqh/Github/_cassandra/cassandra
➜  cassandra git:(cassandra-2.1) ✗ ant generate-idea-files

打开IDEA，导入项目：File>New选择cassandra文件夹，Ant编译

cass1

在IDEA中运行：

1
2
3
4
5
6
7
8
9
10
-Dcassandra-foreground=yes 
-Dcassandra.config=file:///Users/zhengqh/Github/_cassandra/cassandra/conf/cassandra.yaml 
-Dcassandra.storagedir=/Users/zhengqh/Github/_cassandra/cassandra/data 
-Dlogback.configurationFile=file:///Users/zhengqh/Github/_cassandra/cassandra/conf/logback.xml 
-Dcassandra.logdir=/Users/zhengqh/Github/_cassandra/cassandra/data/logs 
-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=7199 
-Dcom.sun.management.jmxremote.local.only=false 
-Dcom.sun.management.jmxremote.authenticate=false 
-Dcom.sun.management.jmxremote.ssl=false 
-ea -Xmx1G

cass2

CQL示例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
➜  cassandra git:(cassandra-2.1) ✗ bin/cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.1.12-SNAPSHOT | CQL spec 3.2.1 | Native protocol v3]
Use HELP for help.
cqlsh> desc keyspaces;

CREATE KEYSPACE mykeyspace  
WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };
USE mykeyspace;  

CREATE TABLE users (
  user_id int PRIMARY KEY,
  fname text,
  lname text
);

INSERT INTO users (user_id,  fname, lname) VALUES (1745, 'john', 'smith');  
INSERT INTO users (user_id,  fname, lname) VALUES (1744, 'john', 'doe');  
INSERT INTO users (user_id,  fname, lname) VALUES (1746, 'john', 'smith');

CREATE INDEX ON users (lname);  
SELECT * FROM users WHERE lname = 'smith';

在IDEA的控制台打印出日志：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
INFO  02:06:06 Create new Keyspace: KSMetaData{name=mykeyspace, strategyClass=SimpleStrategy, strategyOptions={replication_factor=1}, cfMetaData={}, durableWrites=true, userTypes=org.apache.cassandra.config.UTMetaData@306fc698}
INFO  02:06:06 Enqueuing flush of schema_keyspaces: 484 (0%) on-heap, 0 (0%) off-heap
INFO  02:06:06 Writing Memtable-schema_keyspaces@1131334912(0.143KiB serialized bytes, 3 ops, 0%/0% of on/off-heap limit)
INFO  02:06:06 Completed flushing /Users/zhengqh/Github/_cassandra/cassandra/data/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-tmp-ka-2-Data.db (0.000KiB) for commitlog position ReplayPosition(segmentId=1465264873661, position=253619)

INFO  02:06:25 Create new ColumnFamily: org.apache.cassandra.config.CFMetaData@5e22ce80[cfId=709b74d0-2c54-11e6-a6b3-09b80d24c2a2,ksName=mykeyspace,cfName=users,cfType=Standard,comparator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type),comment=,readRepairChance=0.0,dcLocalReadRepairChance=0.1,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.Int32Type,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata=[ColumnDefinition{name=lname, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, componentIndex=0, indexName=null, indexType=null}, ColumnDefinition{name=user_id, type=org.apache.cassandra.db.marshal.Int32Type, kind=PARTITION_KEY, componentIndex=null, indexName=null, indexType=null}, ColumnDefinition{name=fname, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, componentIndex=0, indexName=null, indexType=null}],compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching={"keys":"ALL", "rows_per_partition":"NONE"},defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=99.0PERCENTILE,droppedColumns={},triggers=[],isDense=false]
INFO  02:06:25 Enqueuing flush of schema_keyspaces: 484 (0%) on-heap, 0 (0%) off-heap
INFO  02:06:25 Writing Memtable-schema_keyspaces@1743874812(0.143KiB serialized bytes, 3 ops, 0%/0% of on/off-heap limit)
INFO  02:06:25 Completed flushing /Users/zhengqh/Github/_cassandra/cassandra/data/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-tmp-ka-3-Data.db (0.000KiB) for commitlog position ReplayPosition(segmentId=1465264873661, position=256744)
INFO  02:06:25 Enqueuing flush of schema_columnfamilies: 5834 (0%) on-heap, 0 (0%) off-heap
INFO  02:06:25 Writing Memtable-schema_columnfamilies@1568127712(1.214KiB serialized bytes, 27 ops, 0%/0% of on/off-heap limit)
INFO  02:06:25 Completed flushing /Users/zhengqh/Github/_cassandra/cassandra/data/data/system/schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697/system-schema_columnfamilies-tmp-ka-2-Data.db (0.000KiB) for commitlog position ReplayPosition(segmentId=1465264873661, position=256744)
INFO  02:06:26 Enqueuing flush of schema_columns: 5538 (0%) on-heap, 0 (0%) off-heap
INFO  02:06:26 Writing Memtable-schema_columns@278821915(0.761KiB serialized bytes, 21 ops, 0%/0% of on/off-heap limit)
INFO  02:06:26 Completed flushing /Users/zhengqh/Github/_cassandra/cassandra/data/data/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-tmp-ka-2-Data.db (0.000KiB) for commitlog position ReplayPosition(segmentId=1465264873661, position=256744)
INFO  02:06:26 Loading org.apache.cassandra.config.CFMetaData@80bbefc[cfId=709b74d0-2c54-11e6-a6b3-09b80d24c2a2,ksName=mykeyspace,cfName=users,cfType=Standard,comparator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type),comment=,readRepairChance=0.0,dcLocalReadRepairChance=0.1,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.Int32Type,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata=[ColumnDefinition{name=lname, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, componentIndex=0, indexName=null, indexType=null}, ColumnDefinition{name=user_id, type=org.apache.cassandra.db.marshal.Int32Type, kind=PARTITION_KEY, componentIndex=null, indexName=null, indexType=null}, ColumnDefinition{name=fname, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, componentIndex=0, indexName=null, indexType=null}],compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching={"keys":"ALL", "rows_per_partition":"NONE"},defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=99.0PERCENTILE,droppedColumns={},triggers=[],isDense=false]
INFO  02:06:26 Initializing mykeyspace.users
WARN  02:06:32 Trigger directory doesn't exist, please create it and try again.

INFO  02:07:01 Update ColumnFamily 'mykeyspace/users' From org.apache.cassandra.config.CFMetaData@80bbefc[cfId=709b74d0-2c54-11e6-a6b3-09b80d24c2a2,ksName=mykeyspace,cfName=users,cfType=Standard,comparator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type),comment=,readRepairChance=0.0,dcLocalReadRepairChance=0.1,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.Int32Type,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata=[ColumnDefinition{name=lname, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, componentIndex=0, indexName=null, indexType=null}, ColumnDefinition{name=user_id, type=org.apache.cassandra.db.marshal.Int32Type, kind=PARTITION_KEY, componentIndex=null, indexName=null, indexType=null}, ColumnDefinition{name=fname, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, componentIndex=0, indexName=null, indexType=null}],compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching={"keys":"ALL", "rows_per_partition":"NONE"},defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=99.0PERCENTILE,droppedColumns={},triggers=[],isDense=false] To org.apache.cassandra.config.CFMetaData@1d7a7976[cfId=709b74d0-2c54-11e6-a6b3-09b80d24c2a2,ksName=mykeyspace,cfName=users,cfType=Standard,comparator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type),comment=,readRepairChance=0.0,dcLocalReadRepairChance=0.1,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.Int32Type,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata=[ColumnDefinition{name=lname, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, componentIndex=0, indexName=users_lname_idx, indexType=COMPOSITES}, ColumnDefinition{name=user_id, type=org.apache.cassandra.db.marshal.Int32Type, kind=PARTITION_KEY, componentIndex=null, indexName=null, indexType=null}, ColumnDefinition{name=fname, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, componentIndex=0, indexName=null, indexType=null}],compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching={"keys":"ALL", "rows_per_partition":"NONE"},defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=99.0PERCENTILE,droppedColumns={},triggers=[],isDense=false]
INFO  02:07:01 Enqueuing flush of schema_keyspaces: 484 (0%) on-heap, 0 (0%) off-heap
INFO  02:07:01 Writing Memtable-schema_keyspaces@1069875044(0.143KiB serialized bytes, 3 ops, 0%/0% of on/off-heap limit)
INFO  02:07:01 Completed flushing /Users/zhengqh/Github/_cassandra/cassandra/data/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-tmp-ka-4-Data.db (0.000KiB) for commitlog position ReplayPosition(segmentId=1465264873661, position=465004)
INFO  02:07:01 Compacting [SSTableReader(path='/Users/zhengqh/Github/_cassandra/cassandra/data/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-4-Data.db'), SSTableReader(path='/Users/zhengqh/Github/_cassandra/cassandra/data/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-1-Data.db'), SSTableReader(path='/Users/zhengqh/Github/_cassandra/cassandra/data/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-3-Data.db'), SSTableReader(path='/Users/zhengqh/Github/_cassandra/cassandra/data/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-2-Data.db')]
INFO  02:07:01 Enqueuing flush of schema_columnfamilies: 5834 (0%) on-heap, 0 (0%) off-heap
INFO  02:07:01 Writing Memtable-schema_columnfamilies@312616385(1.214KiB serialized bytes, 27 ops, 0%/0% of on/off-heap limit)
INFO  02:07:01 Completed flushing /Users/zhengqh/Github/_cassandra/cassandra/data/data/system/schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697/system-schema_columnfamilies-tmp-ka-3-Data.db (0.000KiB) for commitlog position ReplayPosition(segmentId=1465264873661, position=465004)
INFO  02:07:01 Enqueuing flush of schema_columns: 1962 (0%) on-heap, 0 (0%) off-heap
INFO  02:07:01 Writing Memtable-schema_columns@310066052(0.267KiB serialized bytes, 7 ops, 0%/0% of on/off-heap limit)
INFO  02:07:01 Completed flushing /Users/zhengqh/Github/_cassandra/cassandra/data/data/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-tmp-ka-3-Data.db (0.000KiB) for commitlog position ReplayPosition(segmentId=1465264873661, position=465004)
INFO  02:07:01 Compacted 4 sstables to [/Users/zhengqh/Github/_cassandra/cassandra/data/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-5,].  704 bytes to 258 (~36% of original) in 39ms = 0.006309MB/s.  5 total partitions merged to 3.  Partition merge counts were {1:2, 3:1, }
INFO  02:07:01 Initializing mykeyspace.users.users_lname_idx
INFO  02:07:01 Enqueuing flush of users: 1681 (0%) on-heap, 0 (0%) off-heap
INFO  02:07:01 Enqueuing flush of users: 0 (0%) on-heap, 0 (0%) off-heap
INFO  02:07:01 Writing Memtable-users@1154357944(0.147KiB serialized bytes, 9 ops, 0%/0% of on/off-heap limit)
INFO  02:07:01 Completed flushing /Users/zhengqh/Github/_cassandra/cassandra/data/data/mykeyspace/users-709b74d02c5411e6a6b309b80d24c2a2/mykeyspace-users-tmp-ka-1-Data.db (0.000KiB) for commitlog position ReplayPosition(segmentId=1465264873661, position=465844)
INFO  02:07:01 Submitting index build of users.users_lname_idx for data in SSTableReader(path='/Users/zhengqh/Github/_cassandra/cassandra/data/data/mykeyspace/users-709b74d02c5411e6a6b309b80d24c2a2/mykeyspace-users-ka-1-Data.db')
INFO  02:07:01 Enqueuing flush of users.users_lname_idx: 1004 (0%) on-heap, 0 (0%) off-heap
INFO  02:07:01 Writing Memtable-users.users_lname_idx@1367019359(0.051KiB serialized bytes, 3 ops, 0%/0% of on/off-heap limit)
INFO  02:07:01 Completed flushing /Users/zhengqh/Github/_cassandra/cassandra/data/data/mykeyspace/users-709b74d02c5411e6a6b309b80d24c2a2/mykeyspace-users.users_lname_idx-tmp-ka-1-Data.db (0.000KiB) for commitlog position ReplayPosition(segmentId=1465264873661, position=465970)
INFO  02:07:01 Enqueuing flush of IndexInfo: 331 (0%) on-heap, 0 (0%) off-heap
INFO  02:07:01 Writing Memtable-IndexInfo@1621801131(0.036KiB serialized bytes, 1 ops, 0%/0% of on/off-heap limit)
INFO  02:07:01 Completed flushing /Users/zhengqh/Github/_cassandra/cassandra/data/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/system-IndexInfo-tmp-ka-1-Data.db (0.000KiB) for commitlog position ReplayPosition(segmentId=1465264873661, position=466071)
INFO  02:07:01 Index build of users.users_lname_idx complete

修改logback.xml的root级别为DEBUG，重启Cassandra，并且启动测试用例CassandraReadTest查询服务： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
DEBUG 03:21:03 Received: STARTUP {CQL_VERSION=3.2.1}, v=3
DEBUG 03:21:03 Received: REGISTER [TOPOLOGY_CHANGE, STATUS_CHANGE, SCHEMA_CHANGE], v=3

DEBUG 03:21:03 Received: QUERY SELECT peer, data_center, rack, tokens, rpc_address, schema_version FROM system.peers, v=3
DEBUG 03:21:03 Received: QUERY SELECT cluster_name, data_center, rack, tokens, partitioner, schema_version FROM system.local WHERE key='local', v=3

DEBUG 03:21:03 Received: QUERY SELECT * FROM system.schema_keyspaces, v=3
DEBUG 03:21:03 Received: QUERY SELECT * FROM system.schema_columnfamilies, v=3
DEBUG 03:21:03 Received: QUERY SELECT * FROM system.schema_columns, v=3
DEBUG 03:21:03 Received: QUERY SELECT * FROM system.schema_usertypes, v=3
DEBUG 03:21:03 Received: QUERY SELECT * FROM system.schema_functions, v=3
DEBUG 03:21:03 Received: QUERY SELECT * FROM system.schema_aggregates, v=3
DEBUG 03:21:03 Received: QUERY SELECT * FROM system.schema_triggers, v=3

DEBUG 03:21:05 Received: QUERY SELECT * FROM forseti.mytab where id='a1', v=3

测试类：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
public class CassandraReadTest {
    public static void main(String[] args) {
        Builder builder = Cluster.builder();
        builder.addContactPoint("127.0.0.1");
        SocketOptions socketOptions = new SocketOptions().setKeepAlive(true).setConnectTimeoutMillis(5 * 10000).setReadTimeoutMillis(100000);
        builder.withSocketOptions(socketOptions);
        Cluster cluster = builder.build();
        Metadata metadata = cluster.getMetadata();
        System.out.printf("Connected to cluster: %s\n", metadata.getClusterName());
        for (Host host : metadata.getAllHosts()) {
            System.out.printf("Datatacenter: %s; Host: %s; Rack: %s\n", host.getDatacenter(), host.getAddress(), host.getRack());
        }
        Session session = cluster.connect();
        ResultSet results = session.execute("SELECT * FROM forseti.mytab where id='a1'");
        for (Row row : results) {
            System.out.println(String.format("%-10s\t%-10s\t%-20s", row.getString("id"), row.getInt("age"), row.getList("names", String.class)));
        }
        cluster.close();
    }
}

准备表和数据：

1
2
3
4
5
6
7
8
9
CREATE KEYSPACE forseti WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };
USE forseti;
CREATE TABLE forseti.mytab (
    id text PRIMARY KEY,
    age int,
    names list<text>
)
insert into mytab(id,age,names)values('a1',30,['01', '02']);
insert into mytab(id,age,names)values('a2',20,['john', 'smith']);

2.2.7版本后，版本为4：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
##创建表
TRACE [SharedPool-Worker-1] 2016-12-28 10:16:44,623 Message.java:506 - Received: QUERY CREATE TABLE forseti.mytab (
   id text PRIMARY KEY,
   age int,
   names list<text>
 )
;, v=4

##插入记录
TRACE [SharedPool-Worker-1] 2016-12-28 10:17:01,909 Message.java:506 - Received: QUERY insert into mytab(id,age,names)values('a1',30,['01', '02']);, v=4
TRACE [SharedPool-Worker-1] 2016-12-28 10:17:01,920 QueryProcessor.java:221 - Process org.apache.cassandra.cql3.statements.UpdateStatement@308a8ddb @CL.ONE
TRACE [SharedPool-Worker-1] 2016-12-28 10:17:01,933 AbstractReplicationStrategy.java:87 - clearing cached endpoints
TRACE [SharedPool-Worker-2] 2016-12-28 10:17:01,950 SlabAllocator.java:143 - 1 regions now allocated in org.apache.cassandra.utils.memory.SlabAllocator@1e9fbb57
TRACE [SharedPool-Worker-1] 2016-12-28 10:17:01,951 Tracing.java:156 - request complete
TRACE [SharedPool-Worker-1] 2016-12-28 10:17:01,951 Message.java:525 - Responding: EMPTY RESULT, v=4

##查询记录
TRACE [SharedPool-Worker-1] 2016-12-28 10:17:28,958 Message.java:506 - Received: QUERY SELECT * FROM forseti.mytab where id='a1', v=4
TRACE [SharedPool-Worker-1] 2016-12-28 10:17:28,959 QueryProcessor.java:221 - Process org.apache.cassandra.cql3.statements.SelectStatement@6e0dee8 @CL.LOCAL_ONE
DEBUG [SharedPool-Worker-1] 2016-12-28 10:17:28,960 SliceQueryPager.java:92 - Querying next page of slice query; new filter: SliceQueryFilter [reversed=false, slices=[[, ]], count=5000, toGroup = 0]
TRACE [SharedPool-Worker-1] 2016-12-28 10:17:28,967 ReadCallback.java:76 - Blockfor is 1; setting up requests to localhost/127.0.0.1
TRACE [SharedPool-Worker-1] 2016-12-28 10:17:28,967 AbstractReadExecutor.java:118 - reading data locally
TRACE [SharedPool-Worker-2] 2016-12-28 10:17:28,992 SliceQueryFilter.java:269 - collecting 0 of 5000: :false:0@1482891421908524
TRACE [SharedPool-Worker-2] 2016-12-28 10:17:28,992 SliceQueryFilter.java:269 - collecting 1 of 5000: age:false:4@1482891421908524
TRACE [SharedPool-Worker-2] 2016-12-28 10:17:28,992 SliceQueryFilter.java:269 - collecting 1 of 5000: names:b7f21510cca311e691854f28b42978f8:false:2@1482891421908524
TRACE [SharedPool-Worker-2] 2016-12-28 10:17:28,992 SliceQueryFilter.java:269 - collecting 1 of 5000: names:b7f21511cca311e691854f28b42978f8:false:2@1482891421908524
TRACE [SharedPool-Worker-1] 2016-12-28 10:17:28,993 StorageProxy.java:1444 - Read: 26 ms.
DEBUG [SharedPool-Worker-1] 2016-12-28 10:17:28,994 AbstractQueryPager.java:95 - Fetched 1 live rows
DEBUG [SharedPool-Worker-1] 2016-12-28 10:17:28,994 AbstractQueryPager.java:112 - Got result (1) smaller than page size (5000), considering pager exhausted
DEBUG [SharedPool-Worker-1] 2016-12-28 10:17:28,994 AbstractQueryPager.java:133 - Remaining rows to page: 2147483646
TRACE [SharedPool-Worker-1] 2016-12-28 10:17:28,994 Tracing.java:156 - request complete
TRACE [SharedPool-Worker-1] 2016-12-28 10:17:28,995 Message.java:525 - Responding: ROWS [id(forseti, mytab), org.apache.cassandra.db.marshal.UTF8Type][age(forseti, mytab), org.apache.cassandra.db.marshal.Int32Type][names(forseti, mytab), org.apache.cassandra.db.marshal.ListType(org.apache.cassandra.db.marshal.UTF8Type)]
 | a1 | 30 | 00000002000000023031000000023032  《= 查询结果（最后一列是List类型）
---, v=4

IDEA中CassandraReadTest的运行结果：

1
2
3
4
5
6
7
INFO  02:17:26 Using native clock to generate timestamps.
WARN  02:17:26 Found Netty's native epoll transport, but not running on linux-based operating system. Using NIO instead.
INFO  02:17:28 Using data-center name 'datacenter1' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor)
INFO  02:17:28 New Cassandra host /127.0.0.1:9042 added
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: /127.0.0.1; Rack: rack1
a1          30          [01, 02]

CQL复杂示例

CQL查询结果

1
2
3
4
5
6
7
8
9
10
11
12
13
14
cqlsh:forseti> select * from velocity_app where attribute='192.168.0.2' and type='ipAddress' and partner_code='tongdun' and app_name='tongdun_app' limit 10;

 attribute   | partner_code | app_name    | type      | sequence_id         | event                                          | timestamp
-------------+--------------+-------------+-----------+---------------------+------------------------------------------------+-----------------
 192.168.0.2 |      tongdun | tongdun_app | ipAddress | 1473427497000-00099 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700099
 192.168.0.2 |      tongdun | tongdun_app | ipAddress | 1473427497000-00098 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700098
 192.168.0.2 |      tongdun | tongdun_app | ipAddress | 1473427497000-00097 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700097
 192.168.0.2 |      tongdun | tongdun_app | ipAddress | 1473427497000-00096 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700096
 192.168.0.2 |      tongdun | tongdun_app | ipAddress | 1473427497000-00095 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700095
 192.168.0.2 |      tongdun | tongdun_app | ipAddress | 1473427497000-00094 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700094
 192.168.0.2 |      tongdun | tongdun_app | ipAddress | 1473427497000-00093 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700093
 192.168.0.2 |      tongdun | tongdun_app | ipAddress | 1473427497000-00092 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700092
 192.168.0.2 |      tongdun | tongdun_app | ipAddress | 1473427497000-00091 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700091
 192.168.0.2 |      tongdun | tongdun_app | ipAddress | 1473427497000-00090 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700090

后台debug日志

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
TRACE [SharedPool-Worker-1] 2016-12-28 10:47:52,234 Message.java:506 - Received: QUERY select * from velocity_app where attribute='192.168.0.2' and type='ipAddress' and partner_code='tongdun' and app_name='tongdun_app' limit 10;, v=4
TRACE [SharedPool-Worker-1] 2016-12-28 10:47:52,236 QueryProcessor.java:221 - Process org.apache.cassandra.cql3.statements.SelectStatement@6fcf961f @CL.ONE
TRACE [SharedPool-Worker-1] 2016-12-28 10:47:52,236 ReadCallback.java:76 - Blockfor is 1; setting up requests to localhost/127.0.0.1
TRACE [SharedPool-Worker-1] 2016-12-28 10:47:52,237 AbstractReadExecutor.java:118 - reading data locally
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,237 SliceQueryFilter.java:269 - collecting 0 of 10: 1473427497000-00099::false:0@1482235780348270
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,238 SliceQueryFilter.java:269 - collecting 1 of 10: 1473427497000-00099:event:false:46@1482235780348270
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,238 SliceQueryFilter.java:269 - collecting 1 of 10: 1473427497000-00099:timestamp:false:8@1482235780348270
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,238 SliceQueryFilter.java:269 - collecting 1 of 10: 1473427497000-00098::false:0@1482235779514638
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,238 SliceQueryFilter.java:269 - collecting 2 of 10: 1473427497000-00098:event:false:46@1482235779514638
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,238 SliceQueryFilter.java:269 - collecting 2 of 10: 1473427497000-00098:timestamp:false:8@1482235779514638
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,238 SliceQueryFilter.java:269 - collecting 2 of 10: 1473427497000-00097::false:0@1482235778858980
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,238 SliceQueryFilter.java:269 - collecting 3 of 10: 1473427497000-00097:event:false:46@1482235778858980
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,238 SliceQueryFilter.java:269 - collecting 3 of 10: 1473427497000-00097:timestamp:false:8@1482235778858980
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,238 SliceQueryFilter.java:269 - collecting 3 of 10: 1473427497000-00096::false:0@1482235778181296
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,238 SliceQueryFilter.java:269 - collecting 4 of 10: 1473427497000-00096:event:false:46@1482235778181296
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,239 SliceQueryFilter.java:269 - collecting 4 of 10: 1473427497000-00096:timestamp:false:8@1482235778181296
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,239 SliceQueryFilter.java:269 - collecting 4 of 10: 1473427497000-00095::false:0@1482235777402787
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,239 SliceQueryFilter.java:269 - collecting 5 of 10: 1473427497000-00095:event:false:46@1482235777402787
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,239 SliceQueryFilter.java:269 - collecting 5 of 10: 1473427497000-00095:timestamp:false:8@1482235777402787
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,239 SliceQueryFilter.java:269 - collecting 5 of 10: 1473427497000-00094::false:0@1482235776625955
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,239 SliceQueryFilter.java:269 - collecting 6 of 10: 1473427497000-00094:event:false:46@1482235776625955
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,239 SliceQueryFilter.java:269 - collecting 6 of 10: 1473427497000-00094:timestamp:false:8@1482235776625955
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,239 SliceQueryFilter.java:269 - collecting 6 of 10: 1473427497000-00093::false:0@1482235775697987
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,239 SliceQueryFilter.java:269 - collecting 7 of 10: 1473427497000-00093:event:false:46@1482235775697987
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,239 SliceQueryFilter.java:269 - collecting 7 of 10: 1473427497000-00093:timestamp:false:8@1482235775697987
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,239 SliceQueryFilter.java:269 - collecting 7 of 10: 1473427497000-00092::false:0@1482235774778831
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,239 SliceQueryFilter.java:269 - collecting 8 of 10: 1473427497000-00092:event:false:46@1482235774778831
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,240 SliceQueryFilter.java:269 - collecting 8 of 10: 1473427497000-00092:timestamp:false:8@1482235774778831
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,240 SliceQueryFilter.java:269 - collecting 8 of 10: 1473427497000-00091::false:0@1482235773836259
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,240 SliceQueryFilter.java:269 - collecting 9 of 10: 1473427497000-00091:event:false:46@1482235773836259
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,240 SliceQueryFilter.java:269 - collecting 9 of 10: 1473427497000-00091:timestamp:false:8@1482235773836259
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,240 SliceQueryFilter.java:269 - collecting 9 of 10: 1473427497000-00090::false:0@1482235773033108
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,240 SliceQueryFilter.java:269 - collecting 10 of 10: 1473427497000-00090:event:false:46@1482235773033108
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,240 SliceQueryFilter.java:269 - collecting 10 of 10: 1473427497000-00090:timestamp:false:8@1482235773033108
**TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,240 SliceQueryFilter.java:269 - collecting 10 of 10: 1473427497000-0009::false:0@1482235712286454**
TRACE [SharedPool-Worker-2] 2016-12-28 10:47:52,240 FileCacheService.java:151 - Estimated memory usage is 65536 compared to actual usage 0
TRACE [SharedPool-Worker-1] 2016-12-28 10:47:52,242 StorageProxy.java:1444 - Read: 5 ms.
TRACE [SharedPool-Worker-1] 2016-12-28 10:47:52,242 Tracing.java:156 - request complete
TRACE [SharedPool-Worker-1] 2016-12-28 10:47:52,242 Message.java:525 - Responding: ROWS [attribute(forseti, velocity_app), org.apache.cassandra.db.marshal.UTF8Type][partner_code(forseti, velocity_app), org.apache.cassandra.db.marshal.UTF8Type][app_name(forseti, velocity_app), org.apache.cassandra.db.marshal.UTF8Type][type(forseti, velocity_app), org.apache.cassandra.db.marshal.UTF8Type][sequence_id(forseti, velocity_app), org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.UTF8Type)][event(forseti, velocity_app), org.apache.cassandra.db.marshal.UTF8Type][timestamp(forseti, velocity_app), org.apache.cassandra.db.marshal.LongType]
 | 192.168.0.2 | tongdun | tongdun_app | ipAddress | 1473427497000-00099 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700099
 | 192.168.0.2 | tongdun | tongdun_app | ipAddress | 1473427497000-00098 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700098
 | 192.168.0.2 | tongdun | tongdun_app | ipAddress | 1473427497000-00097 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700097
 | 192.168.0.2 | tongdun | tongdun_app | ipAddress | 1473427497000-00096 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700096
 | 192.168.0.2 | tongdun | tongdun_app | ipAddress | 1473427497000-00095 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700095
 | 192.168.0.2 | tongdun | tongdun_app | ipAddress | 1473427497000-00094 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700094
 | 192.168.0.2 | tongdun | tongdun_app | ipAddress | 1473427497000-00093 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700093
 | 192.168.0.2 | tongdun | tongdun_app | ipAddress | 1473427497000-00092 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700092
 | 192.168.0.2 | tongdun | tongdun_app | ipAddress | 1473427497000-00091 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700091
 | 192.168.0.2 | tongdun | tongdun_app | ipAddress | 1473427497000-00090 | {accountLogin:zqhxuyuan,ipAddress:192.168.0.2} | 147342749700090
---, v=4

SliceQueryFilter会查询指定key的列，velocity_app表的一行记录有三个列（一个主键，两个普通列：），

1
2
3
collecting 0 of 10: 1473427497000-00099::false:0@1482235780348270
collecting 1 of 10: 1473427497000-00099:event:false:46@1482235780348270
collecting 1 of 10: 1473427497000-00099:timestamp:false:8@1482235780348270

实际的存储结构如下：

partition key column#1 column#2 column#3
192.168.0.2:tongdun:tongdun_app:ipAddress 1473427497000-00099: 1473427497000-00099:event 1473427497000-00099:timestamp
{accountLogin:zqhxuyuan,ipAddress:192.168.0.2} 147342749700099

虽然最后也查询了1473427497000-0009，但是最后的结果并没有这个sequence_id

1
collecting 10 of 10: 1473427497000-0009::false:0@1482235712286454

关于build失败的问题

更新了版本后，重新编译失败：

1
2
3
4
5
6
7
8
9
10
11
12
13
[javac] /Users/zhengqh/Github/_cassandra/cassandra/src/gen-java/org/apache/cassandra/cql/CqlParser.java:13: 错误: 找不到符号
[javac]     import static org.apache.cassandra.cql.AlterTableStatement.OperationType;
[javac]                                           ^
[javac]   符号:   类 AlterTableStatement
[javac]   位置: 程序包 org.apache.cassandra.cql
[javac] /Users/zhengqh/Github/_cassandra/cassandra/src/gen-java/org/apache/cassandra/cql/CqlParser.java:13: 错误: 仅从类和接口静态导入
[javac]     import static org.apache.cassandra.cql.AlterTableStatement.OperationType;
[javac]     ^
[javac] /Users/zhengqh/Github/_cassandra/cassandra/src/gen-java/org/apache/cassandra/cql/CqlParser.java:186: 错误: 找不到符号
[javac]         public void validateMinusSupplied(Object op, final Term value, IntStream stream) throws MissingTokenException
[javac]                                                            ^
[javac]   符号:   类 Term
[javac]   位置: 类 CqlParser

解决办法，先realclean，然后重新编译。注意realclean不仅会删除build文件夹，也会删除gen-java!
http://stackoverflow.com/questions/31518640/unable-to-build-cassandra-source

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
➜  cassandra git:(cassandra-2.2) ant realclean
Buildfile: /Users/zhengqh/Github/_cassandra/cassandra/build.xml

clean:
   [delete] Deleting directory /Users/zhengqh/Github/_cassandra/cassandra/build/test
   [delete] Deleting directory /Users/zhengqh/Github/_cassandra/cassandra/build/classes
   [delete] Deleting directory /Users/zhengqh/Github/_cassandra/cassandra/src/gen-java
   [delete] Deleting directory /Users/zhengqh/Github/_cassandra/cassandra/src/resources/org/apache/cassandra/config
   [delete] Deleting directory /Users/zhengqh/Github/_cassandra/cassandra/build/jacoco

realclean:
   [delete] Deleting directory /Users/zhengqh/Github/_cassandra/cassandra/build

BUILD SUCCESSFUL
Total time: 1 second
➜  cassandra git:(cassandra-2.2) ant generate-idea-files
Buildfile: /Users/zhengqh/Github/_cassandra/cassandra/build.xml

init:
    [mkdir] Created dir: /Users/zhengqh/Github/_cassandra/cassandra/build/classes/main
    [mkdir] Created dir: /Users/zhengqh/Github/_cassandra/cassandra/build/classes/thrift
    [mkdir] Created dir: /Users/zhengqh/Github/_cassandra/cassandra/build/test/lib
    [mkdir] Created dir: /Users/zhengqh/Github/_cassandra/cassandra/build/test/classes
    [mkdir] Created dir: /Users/zhengqh/Github/_cassandra/cassandra/src/gen-java
    [mkdir] Created dir: /Users/zhengqh/Github/_cassandra/cassandra/build/lib
    [mkdir] Created dir: /Users/zhengqh/Github/_cassandra/cassandra/build/jacoco
    [mkdir] Created dir: /Users/zhengqh/Github/_cassandra/cassandra/build/jacoco/partials

maven-ant-tasks-localrepo:
     [copy] Copying 1 file to /Users/zhengqh/Github/_cassandra/cassandra/build

maven-ant-tasks-download:

maven-ant-tasks-init:

maven-declare-dependencies:

maven-ant-tasks-retrieve-build:
[artifact:dependencies] Building ant file: /Users/zhengqh/Github/_cassandra/cassandra/build/build-dependencies.xml
[artifact:dependencies] Building ant file: /Users/zhengqh/Github/_cassandra/cassandra/build/build-dependencies-sources.xml
     [copy] Copying 65 files to /Users/zhengqh/Github/_cassandra/cassandra/build/lib/jars
     [copy] Copying 41 files to /Users/zhengqh/Github/_cassandra/cassandra/build/lib/sources
     [copy] Copying 25 files to /Users/zhengqh/Github/_cassandra/cassandra/build/lib/jars
    [unzip] Expanding: /Users/zhengqh/Github/_cassandra/cassandra/build/lib/jars/org.jacoco.agent-0.7.5.201505241946.jar into /Users/zhengqh/Github/_cassandra/cassandra/build/lib/jars

check-gen-cql3-grammar:

gen-cql3-grammar:
     [echo] Building Grammar /Users/zhengqh/Github/_cassandra/cassandra/src/java/org/apache/cassandra/cql3/Cql.g  ...

generate-cql-html:

build-project:
     [echo] apache-cassandra: /Users/zhengqh/Github/_cassandra/cassandra/build.xml
    [javac] Compiling 45 source files to /Users/zhengqh/Github/_cassandra/cassandra/build/classes/thrift
    [javac] 警告: 来自注释处理程序 'org.openjdk.jmh.generators.BenchmarkProcessor' 的受支持 source 版本 'RELEASE_6' 低于 -source '1.7'
    [javac] 注: /Users/zhengqh/Github/_cassandra/cassandra/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java使用或覆盖了已过时的 API。
    [javac] 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
    [javac] 注: 某些输入文件使用了未经检查或不安全的操作。
    [javac] 注: 有关详细信息, 请使用 -Xlint:unchecked 重新编译。
    [javac] 1 个警告
    [javac] Compiling 1165 source files to /Users/zhengqh/Github/_cassandra/cassandra/build/classes/main
    [javac] 注: Processing compiler hints annotations
    [javac] 警告: 来自注释处理程序 'org.openjdk.jmh.generators.BenchmarkProcessor' 的受支持 source 版本 'RELEASE_6' 低于 -source '1.7'
    [javac] 注: Processing compiler hints annotations
    [javac] 注: Writing compiler command file at META-INF/hotspot_compiler
    [javac] 注: Done processing compiler hints annotations
    [javac] 注: 某些输入文件使用或覆盖了已过时的 API。
    [javac] 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
    [javac] 注: 某些输入文件使用了未经检查或不安全的操作。
    [javac] 注: 有关详细信息, 请使用 -Xlint:unchecked 重新编译。
    [javac] 1 个警告

createVersionPropFile:
    [mkdir] Created dir: /Users/zhengqh/Github/_cassandra/cassandra/src/resources/org/apache/cassandra/config
[propertyfile] Creating new property file: /Users/zhengqh/Github/_cassandra/cassandra/src/resources/org/apache/cassandra/config/version.properties
     [copy] Copying 2 files to /Users/zhengqh/Github/_cassandra/cassandra/build/classes/main
     [copy] Copying 1 file to /Users/zhengqh/Github/_cassandra/cassandra/conf

build:

build-test:
    [javac] Compiling 338 source files to /Users/zhengqh/Github/_cassandra/cassandra/build/test/classes
    [javac] 警告: 来自注释处理程序 'org.openjdk.jmh.generators.BenchmarkProcessor' 的受支持 source 版本 'RELEASE_6' 低于 -source '1.7'
    [javac] 注: 某些输入文件使用或覆盖了已过时的 API。
    [javac] 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
    [javac] 注: 某些输入文件使用了未经检查或不安全的操作。
    [javac] 注: 有关详细信息, 请使用 -Xlint:unchecked 重新编译。
    [javac] 1 个警告
     [copy] Copying 12 files to /Users/zhengqh/Github/_cassandra/cassandra/build/test/classes

generate-idea-files:
     [copy] Copying 1 file to /Users/zhengqh/Github/_cassandra/cassandra/.idea
     [copy] Copying 1 file to /Users/zhengqh/Github/_cassandra/cassandra

BUILD SUCCESSFUL
Total time: 1 minute 5 seconds

本文标题:Cassandra源码分析

文章作者:任何忧伤,都抵不过世界的美丽

发布时间:2016年06月07日 - 00时00分

最后更新:2019年02月14日 - 21时42分

原始链接:http://github.com/zqhxuyuan/2016/06/07/Cassandra-Code/

许可协议: "署名-非商用-相同方式共享 3.0" 转载请保留原文链接及作者。


文章目录
  1. 1. IDEA中运行Cassandra-2.1
  2. 2. CQL示例
  3. 3. CQL复杂示例
  4. 4. 关于build失败的问题