drrtuy`s board

янв 21, 2019

How to save time working with MariaDB Columnstore code.

I work a lot with MariaDB Columnstore source code and I used to spent much time just starting/restarting MCS. I usually don't need a cluster but a single server installation so there are ways to optimize starting/restarting routine. Before this commit cluster restart took about 3 minutes and now it takes only 6 seconds. Moreover I can get my stdout,stderr output without much trouble now because the streams goes neatly into the place that I configured. For details look here. Today I will share a number of scripts I use to speed re-/starting routine up but all credits for huge savings of the precious time goes to the author of the commit. And by the way here is the repo you could grab the scripts from.

Presetup

All the scripts mentioned here use a number of variables that must be set for your environment. The variables are in the very beginning of the scripts.

Initial installation

I use run_cs_with_oam_skiped for the purpose. A word of warning this one DELETES the MCS installation. The logic behind it simple: it compiles/installs both the server and MCS engine code and calls a number of scripts creating initial system catalog. I advise you to go through commentaries of the script itself before you run it. Here are the interesting variables that one must set before he can use the script. And yes, I omit self-explanatory variables:

  • MCS_TMP_DIR=/tmp/columnstore_tmp_files ( This one points to a directory that contains temporary columnstore installation files. It shouldn't be changed. )
  • MCS_SCRIPTS_REPO_PREFIX=/git/cs-docker-tools ( This one points to top directory of the scripts repository

The example output for this script is very long so I also omit it. The ps output is different when you run MCS using the script.

root@4bbf704f05b8:/git/mariadb-columnstore-server/d1-mariadb-columnstore-engine# ps -auxfw
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     29369  0.0  0.0  20288  2756 pts/0    Ss   янв16   0:00 bash
root      3293  0.0  0.0  38372  3536 pts/0    R+   09:13   0:00  \_ ps -auxfw
root         1  0.0  0.0  20048  1824 ?        Ss   янв14   0:05 /bin/bash /scripts/idle.sh
syslog   22073  0.0  0.0 263036  3784 ?        Ssl  янв18   0:00 /usr/sbin/rsyslogd
root      3056  0.0  0.2 12226744 41860 pts/0  S<l  08:29   0:00 /usr/local/mariadb/columnstore/bin/PrimProc
root      3060  0.0  0.0 907272 12360 pts/0    Sl   08:29   0:00 /usr/local/mariadb/columnstore/bin/controllernode
root      3065  0.0  0.1 275408 18176 pts/0    Sl   08:29   0:00 /usr/local/mariadb/columnstore/bin/workernode DBRM_Worker1
root      3112  0.0  0.2 1202380 32696 pts/0   S<l  08:29   0:00 /usr/local/mariadb/columnstore/bin/ExeMgr
root      3115  0.0  0.2 423020 40160 pts/0    Sl   08:29   0:00 /usr/local/mariadb/columnstore/bin/WriteEngineServer
root      3142  0.0  0.2 428416 43736 pts/0    Sl   08:29   0:00 /usr/local/mariadb/columnstore/bin/DMLProc
root      3145  0.0  0.2 279164 36396 pts/0    Sl   08:29   0:00 /usr/local/mariadb/columnstore/bin/DDLProc
mysql     3148  0.0  2.5 2961452 418788 pts/0  Sl   08:29   0:01 /usr/local/mariadb/columnstore/mysql/bin/mysqld --basedir=/usr/local/mariadb/columnstore/mysql/ --datadir=/usr/local/mariadb/columnstore/mysql/db --plugin-dir=/usr/local/mar
root      3291  0.0  0.0   6176   848 ?        S    09:13   0:00 sleep 60

Fast reinstall

If I only add some debugging output, it isn't worth it to delete / reinstall the whole MCS; better to stop MCS, call make install, and then start MCS again. I use sreinstall for this. It gracefully stops MCS, compiles and installs from the preset source directory and then starts MCS. This method works only if I installed MCS using the method described in the previous section. Here is the example that I minimized.

root@4bbf704f05b8:/git/mariadb-columnstore-server/d1-mariadb-columnstore-engine# sreinstall 1
Saved to /usr/local/mariadb/columnstore/data1/systemFiles/dbrm/BRM_saves
Sending SIGTERM
2019-01-21  9:18:15 0 [Note] /usr/local/mariadb/columnstore/mysql/bin/mysqld (initiated by: unknown): Normal shutdown
2019-01-21  9:18:15 0 [Note] InnoDB: FTS optimize thread exiting.
2019-01-21  9:18:15 0 [Note] Event Scheduler: Purging the queue. 0 events
2019-01-21  9:18:15 0 [Note] InnoDB: Starting shutdown...
...
-- Up-to-date: /usr/local/mariadb/columnstore/bin/WriteEngineServer
-- Up-to-date: /usr/local/mariadb/columnstore/bin/cpimport.bin
-- Up-to-date: /usr/local/mariadb/columnstore/bin/cpimport
This is dumb. DMLProc seems to need it.
Starting...
loading BRM from /usr/local/mariadb/columnstore/data1/systemFiles/dbrm/BRM_saves
OK.
Successfully loaded BRM snapshot
Successfully replayed 0 BRM transactions
workernode PID = 3909
controllernode PID = 3912
PrimProc PID = 3915
ExeMgr PID = 3971
WriteEngineServer PID = 3974
DMLProc PID = 4002
DDLProc PID = 4005
2019-01-21  9:18:21 0 [Note] /usr/local/mariadb/columnstore/mysql/bin/mysqld (mysqld 10.3.11-MariaDB-debug) starting as process 4008 ...
190121  9:18:21 Columnstore: Started; Version: 1.2.2-1
190121  9:18:21 Columnstore: Started; Version: 1.2.2-1
2019-01-21  9:18:21 0 [Note] InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!!
2019-01-21  9:18:21 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-01-21  9:18:21 0 [Note] InnoDB: Uses event mutexes
2019-01-21  9:18:21 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2019-01-21  9:18:21 0 [Note] InnoDB: Number of pools: 1
2019-01-21  9:18:21 0 [Note] InnoDB: Using SSE2 crc32 instructions
2019-01-21  9:18:21 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2019-01-21  9:18:21 0 [Note] InnoDB: Completed initialization of buffer pool
2019-01-21  9:18:21 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2019-01-21  9:18:21 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
2019-01-21  9:18:21 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2019-01-21  9:18:21 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2019-01-21  9:18:21 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2019-01-21  9:18:21 0 [Note] InnoDB: Waiting for purge to start
2019-01-21  9:18:21 0 [Note] InnoDB: 10.3.11 started; log sequence number 1656714; transaction id 49
2019-01-21  9:18:21 0 [Note] InnoDB: Loading buffer pool(s) from /usr/local/mariadb/columnstore/mysql/db/ib_buffer_pool
2019-01-21  9:18:21 0 [Note] InnoDB: Buffer pool(s) load completed at 190121  9:18:21
2019-01-21  9:18:21 0 [Note] Plugin 'FEEDBACK' is disabled.
2019-01-21  9:18:21 0 [Note] Server socket created on IP: '::'.
2019-01-21  9:18:21 0 [Note] Reading of all Master_info entries succeded
2019-01-21  9:18:21 0 [Note] Added new Master_info '' to hash table
2019-01-21  9:18:21 0 [Note] /usr/local/mariadb/columnstore/mysql/bin/mysqld: ready for connections.
Version: '10.3.11-MariaDB-debug'  socket: '/usr/local/mariadb/columnstore/mysql/lib/mysql/mysql.sock'  port: 3306  Columnstore 1.2.3-1

Start/stop

Let's say an ExeMgr has crashed and I don't want to reinstall but just to gracefully restart MCS. I use a pair: startcs, stopcs for this. And as I said before I can have my stdout, stderr output now and startcs has the variable(LOG_PREFIX) that points to a directory where all logging goes. Here is an example output.

root@4bbf704f05b8:/git/mariadb-columnstore-server/d1-mariadb-columnstore-engine# stopcs
Saved to /usr/local/mariadb/columnstore/data1/systemFiles/dbrm/BRM_saves
Sending SIGTERM
2019-01-21  9:25:21 0 [Note] /usr/local/mariadb/columnstore/mysql/bin/mysqld (initiated by: unknown): Normal shutdown
2019-01-21  9:25:21 0 [Note] InnoDB: FTS optimize thread exiting.
2019-01-21  9:25:21 0 [Note] Event Scheduler: Purging the queue. 0 events
2019-01-21  9:25:21 0 [Note] InnoDB: Starting shutdown...
2019-01-21  9:25:21 0 [Note] InnoDB: Dumping buffer pool(s) to /usr/local/mariadb/columnstore/mysql/db/ib_buffer_pool
2019-01-21  9:25:21 0 [Note] InnoDB: Buffer pool(s) dump completed at 190121  9:25:21
2019-01-21  9:25:23 0 [Note] InnoDB: Shutdown completed; log sequence number 1656723; transaction id 50
2019-01-21  9:25:23 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2019-01-21  9:25:23 0 [Note] /usr/local/mariadb/columnstore/mysql/bin/mysqld: Shutdown complete

Sending SIGKILL
Clearing SHM
root@4bbf704f05b8:/git/mariadb-columnstore-server/d1-mariadb-columnstore-engine# startcs
This is dumb. DMLProc seems to need it.
Starting...
loading BRM from /usr/local/mariadb/columnstore/data1/systemFiles/dbrm/BRM_saves
OK.
Successfully loaded BRM snapshot
Successfully replayed 0 BRM transactions
workernode PID = 4094
controllernode PID = 4097
PrimProc PID = 4100
ExeMgr PID = 4156
WriteEngineServer PID = 4159
DMLProc PID = 4186
DDLProc PID = 4189
2019-01-21  9:25:29 0 [Note] /usr/local/mariadb/columnstore/mysql/bin/mysqld (mysqld 10.3.11-MariaDB-debug) starting as process 4192 ...
190121  9:25:29 Columnstore: Started; Version: 1.2.2-1
190121  9:25:29 Columnstore: Started; Version: 1.2.2-1
2019-01-21  9:25:29 0 [Note] InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!!
2019-01-21  9:25:29 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-01-21  9:25:29 0 [Note] InnoDB: Uses event mutexes
2019-01-21  9:25:29 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2019-01-21  9:25:29 0 [Note] InnoDB: Number of pools: 1
2019-01-21  9:25:29 0 [Note] InnoDB: Using SSE2 crc32 instructions
2019-01-21  9:25:29 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2019-01-21  9:25:29 0 [Note] InnoDB: Completed initialization of buffer pool
2019-01-21  9:25:29 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2019-01-21  9:25:30 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
2019-01-21  9:25:30 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2019-01-21  9:25:30 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2019-01-21  9:25:30 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2019-01-21  9:25:30 0 [Note] InnoDB: Waiting for purge to start
2019-01-21  9:25:30 0 [Note] InnoDB: 10.3.11 started; log sequence number 1656723; transaction id 49
2019-01-21  9:25:30 0 [Note] InnoDB: Loading buffer pool(s) from /usr/local/mariadb/columnstore/mysql/db/ib_buffer_pool
2019-01-21  9:25:30 0 [Note] InnoDB: Buffer pool(s) load completed at 190121  9:25:30
root@4bbf704f05b8:/git/mariadb-columnstore-server/d1-mariadb-columnstore-engine# 2019-01-21  9:25:30 0 [Note] Plugin 'FEEDBACK' is disabled.
2019-01-21  9:25:30 0 [Note] Server socket created on IP: '::'.
2019-01-21  9:25:30 0 [Note] Reading of all Master_info entries succeded
2019-01-21  9:25:30 0 [Note] Added new Master_info '' to hash table
2019-01-21  9:25:30 0 [Note] /usr/local/mariadb/columnstore/mysql/bin/mysqld: ready for connections.
Version: '10.3.11-MariaDB-debug'  socket: '/usr/local/mariadb/columnstore/mysql/lib/mysql/mysql.sock'  port: 3306  Columnstore 1.2.3-1

Conclusion

This handful of scripts helps me to speed up the development. Hope you find them useful too.

Click to read and post comments