sqoop、sqoop2介绍及如何使用

请问一下sqoop2怎么使用sql语句去过滤数据再导入到hdfs

学习啦

楼主我的sqoop1.99.4怎么报这个错呀，检查配置什么的都是呀。。
我安装的hadoop2.7.1 是不兼容的原因吗？？
[root@hadoop21 bin]# ./sqoop.sh server start
Sqoop home directory: /opt/sqoop-1.99.4
Setting SQOOP_HTTP_PORT:     12000
Setting SQOOP_ADMIN_PORT:     12001
Using   CATALINA_OPTS:      
Adding to CATALINA_OPTS:    -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001
Using CATALINA_BASE:   /opt/sqoop-1.99.4/server
Using CATALINA_HOME:   /opt/sqoop-1.99.4/server
Using CATALINA_TMPDIR: /opt/sqoop-1.99.4/server/temp
Using JRE_HOME:        /opt/jdk1.7.0_45
Using CLASSPATH:       /opt/sqoop-1.99.4/server/bin/bootstrap.jar
[root@hadoop21 bin]# ./sqoop.sh client
Sqoop home directory: /opt/sqoop-1.99.4
Sqoop Shell: Type 'help' or '\h' for help.

sqoop:000> show version --all
client version:
  Sqoop 1.99.4 source revision 2475a76ef70a0660f381c75c3d47d0d24f00b57f
  Compiled by gshapira on Sun Nov 16 02:50:00 PST 2014
Exception has occurred during processing command
Exception: com.sun.jersey.api.client.ClientHandlerException Message: java.net.ConnectException: Connection refused

hadoop2.4 应该用哪个版本的sqoop

学习学习

不错 感谢分享！

楼主，你好，执行sqoop import的时候报错说生成的的那个class not found.
执行的命令是sqoop import命令是：
sqoop import --connect jdbc:mysql://192.168.7.225:3306/mas --username mas --password password --table MAS_EC_USER_INFO -m 1
其中MAS_EC_USER_INFO是要查询表的名字
下面是报错信息：
16/03/16 21:33:18 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
16/03/16 21:33:18 INFO mapreduce.Job: Running job: job_local1327636099_0001
16/03/16 21:33:18 INFO mapred.LocalJobRunner: OutputCommitter set in config null
16/03/16 21:33:18 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
16/03/16 21:33:18 INFO mapred.LocalJobRunner: Waiting for map tasks
16/03/16 21:33:18 INFO mapred.LocalJobRunner: Starting task: attempt_local1327636099_0001_m_000000_0
16/03/16 21:33:19 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
16/03/16 21:33:19 INFO db.DBInputFormat: Using read commited transaction isolation
16/03/16 21:33:19 INFO mapred.MapTask: Processing split: 1=1 AND 1=1
16/03/16 21:33:19 INFO mapred.LocalJobRunner: map task executor complete.
16/03/16 21:33:19 WARN mapred.LocalJobRunner: job_local1327636099_0001
java.lang.Exception: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class MAS_EC_USER_INFO not found
        at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class MAS_EC_USER_INFO not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2154)
        at org.apache.sqoop.mapreduce.db.DBConfiguration.getInputClass(DBConfiguration.java:403)
        at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.createDBRecordReader(DataDrivenDBInputFormat.java:237)
        at org.apache.sqoop.mapreduce.db.DBInputFormat.createRecordReader(DBInputFormat.java:263)
        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>(MapTask.java:512)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:755)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: Class MAS_EC_USER_INFO not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2060)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2152)
        ... 12 more
16/03/16 21:33:19 INFO mapreduce.Job: Job job_local1327636099_0001 running in uber mode : false
16/03/16 21:33:19 INFO mapreduce.Job:  map 0% reduce 0%
16/03/16 21:33:19 INFO mapreduce.Job: Job job_local1327636099_0001 failed with state FAILED due to: NA
16/03/16 21:33:19 INFO mapreduce.Job: Counters: 0
16/03/16 21:33:19 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
16/03/16 21:33:19 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 3.3336 seconds (0 bytes/sec)
16/03/16 21:33:19 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
16/03/16 21:33:19 INFO mapreduce.ImportJobBase: Retrieved 0 records.
16/03/16 21:33:19 ERROR tool.ImportTool: Error during import: Import job failed!

ezliqi 发表于 2016-3-16 21:36
楼主，你好，执行sqoop import的时候报错说生成的的那个class not found.
执行的命令是sqoop import命令是 ...

已经解决
在mapred-site.xml里加上下面配置就可以了
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>

create connection --cid 1     这个命令是不是写错了，应该是create link --cid 1

过来那分的。 哎真难