读书人

Hive配备参考

发布时间: 2013-01-28 11:49:56 作者: rapoo

Hive配置参考
安装环境:?? ?

? ? ? 机器 只需要安装一台机器
? ?? ?操作系统:Ubuntu 11.04 64操作系统
? ?? ?hadoop:版本是1.0.2,安装在/usr/local/hadoop
? ?? ?sun jdk:版本是1.6.0_31 64bit,安装在/usr/local/jdk
? ?? ?hive:版本是0.8.1,安装在/usr/local/hive

安装步骤:1.下载??

? ? 下载hive:http://labs.mop.com/apache-mirror/hive/hive-0.8.1/hive-0.8.1.tar.gz


2.安装

(1)上传hive安装包到机器上,使用root用户登陆:? ?

<property>  <name>hive.metastore.local</name>  <value>false</value>  <description>controls whether to connect to remove metastore server or open a new metastore server in Hive Client JVM</description></property><property>  <name>javax.jdo.option.ConnectionURL</name>  <value>jdbc:mysql://mysql_server_host:3306/hivedb?createDatabaseIfNotExist=true&useUnicode=true&characterEncoding=latin1</value>  <description>JDBC connect string for a JDBC metastore</description></property><property>  <name>javax.jdo.option.ConnectionDriverName</name>  <value>com.mysql.jdbc.Driver</value>  <description>Driver class name for a JDBC metastore</description></property><property>  <name>javax.jdo.option.ConnectionUserName</name>  <value>mysql_username</value>  <description>username to use against metastore database</description></property><property>  <name>javax.jdo.option.ConnectionPassword</name>  <value>mysql_password</value>  <description>password to use against metastore database</description></property><property>  <name>hive.stats.dbconnectionstring</name>  <value>jdbc:mysql://mysql_server_host:3306/hive_stats?useUnicode=true&characterEncoding=latin1&user=mysql_username&password=mysql_password&createDatabaseIfNotExist=true</value>  <description>The default connection string for the database that stores temporary hive statistics.</description></property><property>  <name>hive.stats.dbconnectionstring</name>  <value>jdbc:mysql://mysql_server_host:3306/hive_stats?useUnicode=true&characterEncoding=utf8&user=mysql_username&password=mysql_password&createDatabaseIfNotExist=true</value>  <description>The default connection string for the database that stores temporary hive statistics.</description></property><property>  <name>hive.stats.dbclass</name>  <value>jdbc:mysql</value>  <description>The default database that stores temporary hive statistics.</description></property><property>  <name>hive.stats.jdbcdriver</name>  <value>com.mysql.jdbc.Driver</value>  <description>The JDBC driver for the database that stores temporary hive statistics.</description></property><property>  <name>hive.metastore.uris</name>  <value>thrift://127.0.0.1:9083</value></property>

添加metastore启动脚本bin/hive-metastore.sh

#!/bin/shnohup ./hive --service metastore >> metastore.log 2>&1 &echo $! > hive-metastore.pid

添加hive server启动脚本bin/hive-server.sh

nohup ./hive --service hiveserver >> hiveserver.log 2>&1 &echo $! > hive-server.pid

启动metastore和hive server

./hive-metastore.sh./hive-server.sh

使用客户端连接和测试

参考之前的hive测试步骤一一执行。

如果出现
FAILED: Error in metadata: javax.jdo.JDODataStoreException: Error(s) were found while auto-creating/validating the datastore for classes. The errors are printed in the log, and are attached to this exception.
NestedThrowables:
com.mysql.jdbc.exceptions.MySQLSyntaxErrorException: Specified key was too long; max key length is 1000 bytes
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

或者是
FAILED: Error in metadata: MetaException(message:Got exception: org.apache.thrift.transport.TTransportException null)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

以上错误,主要是在select操作、drop操作时,就会出错。

这样的错误,可以修改hivedb数据库编码:

mysql> alter database hivedb character set latin1;

然后重启hive metastore和hive server就可以了

mysql数据库的编码目前最好设置为latin1,否则使用hive会出现莫名其妙的问题

?

参考URL:

http://www.cnblogs.com/ggjucheng/archive/2012/04/26/2471341.html

读书人网 >开源软件

热点推荐