五月 29th, 2010
infobright 使用心得
一、安装:
去官方网站下载rpm包(rpm安装方便,当然也可以源码安装,这个随自己的喜好。)
1.安装rpm包:
[root@xiong ~]# wget http://www.infobright.org/downloads/ice/infobright-3.3.2-x86_64-ice.rpm [root@xiong ~]# rpm -ivh infobright-3.3.2-x86_64-ice.rpm [root@xiong ~]# rpm -ivh infobright-3.3.2-x86_64-ice.rpm Preparing... ########################################### [100%] Installing infobright 3.3.2 x86_64 The installer will generate /tmp/ib3.3.2-install.log install trace log. 1:infobright ########################################### [100%] Creating/Updating datadir and cachedir Creating user mysql and group mysql Installing default databases Installing MySQL system tables... OK Filling help tables... OK ...... #省略 ...... System Physical memory: 128(MB) System memory too low! Infobright optimal ServerMainHeapSize is set to 400(MB) Infobright optimal ServerCompressedHeapSize is set to 200(MB) Infobright optimal LoaderMainHeapSize is set to 300(MB) Infobright server installed into folder /usr/local/infobright Installation log file /tmp/ib3.3.2-install.log -------------------------------------- To activate infobright server, please run ./postconfig.sh script from /usr/local/infobright-3.3.2-x86_64. Example command: cd /usr/local/infobright-3.3.2-x86_64; ./postconfig.sh #安装成功。
2.启动:
cd /usr/local/infobright && /usr/local/infobright/bin/mysqld_safe & [root@xiong ~]# ps aux |grep infobright root 5049 0.0 0.9 8672 1208 ttyp2 S 00:21 0:00 /bin/sh /usr/local/infobright/bin/mysqld_safe mysql 5097 0.3 6.8 652860 8948 ttyp2 Sl 00:21 0:00 /usr/local/infobright/bin/mysqld --basedir=/usr/local/infobright --datadir=/usr/local/infobright/data --user=mysql --log-error=/usr/local/infobright/data/xiong.err --pid-file=/usr/local/infobright/data/xiong.pid root 5101 0.0 0.4 6048 592 ttyp2 R+ 00:21 0:00 grep infobright #ok,启动成功了,下面我们先修改root密码。 [root@xiong ~]# /usr/local/infobright/bin/mysqladmin -u root password '12312345' [root@xiong ~]# /usr/local/infobright/bin/mysql -uroot -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.1.40 build number (revision)=IB_3.3.2_r7501_7556(ice) Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> #安装就算完成了,如果要创建新账户和分配权限这和mysql操作方式是一样的如: GRANT ALL PRIVILEGES ON *.* TO 'dbuser'@'localhost' IDENTIFIED BY '123123';
二、使用:
注意:
1.ICE版只支持 “LOAD DATA INFILE”的方式导入数据,不支持INSERT、UPDATE、DELETE, 并且只支持10多个并发查询。
2.另外目前版本linux版本没有gbk字符集,不过支持utf8的,如果要支持gbk的,自己要去找找资料(用SHOW CHARACTER SET;查看字符集)。
3.BRIGHTHOUSE引擎建表不能有AUTO_INCREMENT,unsigned、unique、PRIMARY KEY、KEY等关键字。
4.不支持MEDIUMTEXT,LONGTEXT ,这2个的话要改成TEXT。
如:
#普通的建表方法 CREATE TABLE test ( id int(11) NOT NULL auto_increment, user_name varchar(20) collate gbk_bin NOT NULL default '', user_password varchar(40) collate gbk_bin NOT NULL default '', login_num int(11) NOT NULL default '0', last_time datetime NOT NULL, comment varchar(255) collate gbk_bin NOT NULL default '', is_closed tinyint(1) unsigned NOT NULL default '0', service tinyint(1) unsigned NOT NULL default '0', PRIMARY KEY (id), UNIQUE KEY admin_name (user_name), KEY service (service) ) ENGINE=InnoDB DEFAULT CHARSET=gbk #infobright 建表 CREATE TABLE test ( id int(11) NOT NULL , user_name varchar(20) NOT NULL , user_password varchar(40) NOT NULL , login_num int(11) NOT NULL , last_time datetime NOT NULL , comment varchar(255) NOT NULL , is_closed tinyint(1) NOT NULL , service tinyint(1) NOT NULL , ) ENGINE=BRIGHTHOUSE DEFAULT CHARSET=utf8 COLLATE=utf8_bin #数据从myisam数据库导出 mysql -S /tmp/mysql.sock -D test -e "SELECT * FROM test WHERE 1 into outfile '/tmp/test.csv' FIELDS TERMINATED BY ',' ENCLOSED BY '\"' ESCAPED BY '\\\' LINES TERMINATED BY '\n';" #到infobright导入 /usr/local/infobright/bin/mysql -S /tmp/mysql-ib.sock -D test -uroot -p --skip-column-names -e "LOAD DATA INFILE '/tmp/test.csv' INTO TABLE admins CHARACTER SET utf8 FIELDS TERMINATED BY ',' ESCAPED BY '\\\' LINES TERMINATED BY '\n';"
基本安装到简单使用就到这里了,为了操作方便以后用可以使用py脚本来管理导入导出。
注:写下面的脚本的啥都不是,就是个小菜
.
#!/bin/env python """ @Author: xiongyq @Desc: export db to csv """ import sys,os import subprocess as SP import optparse import re import platform def retable (table, out_retable): regex=ur"%s" % out_retable match = re.search(regex, table) if match: return True else: return False p = optparse.OptionParser() p.add_option('--only_table', '-o', default=False, help="only export this table to csv.",action="store_true") p.add_option('--except_table', '-e', default=False, help="besides these tables to csv.",action="store_true") p.add_option('--except_retable', '-r', default=False, help="use regular besides these tables to csv.",action="store_true") op, args = p.parse_args() if platform.system() == 'Windows':sys.exit('Don\'t support Windows platforms.') #first is_only_table = op.only_table #second is_except_table = op.except_table #third is_except_retable = op.except_retable database = 'test' to_path = '/tmp/csv/' + database out_table = ['table1',] out_retable = 'table_2010' only_table = ['table2','table3',] if os.path.exists(to_path) == False: oldmask = os.umask(0) os.makedirs(to_path, 0777) os.umask(oldmask) p=SP.Popen("/usr/bin/mysql -S /tmp/mysql.sock -e \"select TABLE_NAME from INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA='%s'\"" % database,shell=True,stdout=SP.PIPE,stderr=SP.PIPE,close_fds=True) stdoutdata, stderrdata=p.communicate() if p.returncode != 0: print 'Error:',stderrdata sys.exit(1) tables = stdoutdata.split("\n") fo = open(to_path + '/table.log','w') for table in tables : table = table.strip() if table.upper() == 'TABLE_NAME': continue if len(table) == 0: continue if is_only_table == True and table in only_table: is_out_table = False is_out_retable = False elif is_only_table == True: continue if is_except_table == True and table in out_table: continue if is_except_retable == True and retable(table, out_retable): continue if is_only_table == False and is_except_table == False and is_except_retable == False: continue command = r'''/usr/bin/mysql -S /tmp/mysql.sock -D %s -e "SELECT * FROM %s WHERE 1 into outfile '%s/%s.csv' FIELDS TERMINATED BY ',' ENCLOSED BY '\"' ESCAPED BY '\\\' LINES TERMINATED BY '\n';" ''' % (database,table,to_path,table) print '-- import %s start...' % table p=SP.Popen(command,shell=True,stdout=SP.PIPE,stderr=SP.PIPE,close_fds=True) std_1, std_2=p.communicate() if p.returncode != 0: print "--\texport csv table %s" % table, std_2 continue fo.write(table + "\n") fo.flush() print '-- import %s end!' % table fo.close() print 'Success!'

五月 29th, 2010 5:41 下午
听说一样的SELECT语句,速度比MyISAM、InnoDB等引擎快5~60倍,如果是这样的话,那值得用用。可惜的是它并发支持不多,只能在后台做数据分析查询。
五月 29th, 2010 5:43 下午
是的,infobright 一般适用来人工做数据分析,人工智能方面。