sqoop：File does not exist:

sqoop从mysql中拉表报错：

02-08-2019 10:47:28 CST bi_cal_resume_achieve_sqoop_import INFO - 19/08/02 10:47:28 INFO hive.metastore: Trying to connect to metastore with URI thrift://pf-bigdata1:9083
02-08-2019 10:47:28 CST bi_cal_resume_achieve_sqoop_import INFO - 19/08/02 10:47:28 INFO hive.metastore: Opened a connection to metastore, current connections: 1
02-08-2019 10:47:28 CST bi_cal_resume_achieve_sqoop_import INFO - 19/08/02 10:47:28 INFO hive.metastore: Connected to metastore.
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 19/08/02 10:47:29 ERROR sqoop.Sqoop: Got exception running Sqoop: org.kitesdk.data.DatasetIOException: Could not read schema
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - org.kitesdk.data.DatasetIOException: Could not read schema
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.kitesdk.data.spi.hive.HiveUtils.descriptorForTable(HiveUtils.java:152)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.kitesdk.data.spi.hive.HiveAbstractMetadataProvider.load(HiveAbstractMetadataProvider.java:104)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.kitesdk.data.spi.filesystem.FileSystemDatasetRepository.load(FileSystemDatasetRepository.java:197)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.kitesdk.data.Datasets.load(Datasets.java:108)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.kitesdk.data.Datasets.load(Datasets.java:165)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.kitesdk.data.Datasets.load(Datasets.java:187)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.sqoop.mapreduce.ParquetJob.configureImportJob(ParquetJob.java:123)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.sqoop.mapreduce.DataDrivenImportJob.configureMapper(DataDrivenImportJob.java:130)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:267)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:127)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:513)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.sqoop.Sqoop.main(Sqoop.java:252)


//主要是这一句
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - Caused by: java.io.FileNotFoundException: File does not exist: /data/hive/warehouse/os.db/os_sh_t_g_account/.metadata/schemas/1.avsc



02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:2094)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2064)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1977)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:575)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java:92)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:376)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2226)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2222)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at java.security.AccessController.doPrivileged(Native Method)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at javax.security.auth.Subject.doAs(Subject.java:422)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2220)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:1289)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1274)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1262)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:307)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:273)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.DFSInputStream.<init>(DFSInputStream.java:265)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1593)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:338)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:334)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:334)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:784)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.kitesdk.data.spi.Schemas.open(Schemas.java:210)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.kitesdk.data.spi.Schemas.fromAvsc(Schemas.java:71)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.kitesdk.data.DatasetDescriptor$Builder.schemaUri(DatasetDescriptor.java:436)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.kitesdk.data.spi.hive.HiveUtils.descriptorForTable(HiveUtils.java:150)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	... 18 more
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - Caused by: org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): File does not exist: /data/hive/warehouse/ods.db/os_sh_t_gw_account/.metadata/schemas/1.avsc
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:2094)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2064)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1977)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:575)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java:92)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:376)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2226)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2222)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at java.security.AccessController.doPrivileged(Native Method)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at javax.security.auth.Subject.doAs(Subject.java:422)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2220)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.ipc.Client.call(Client.java:1504)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.ipc.Client.call(Client.java:1441)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at com.sun.proxy.$Proxy10.getBlockLocations(Unknown Source)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getBlockLocations(ClientNamenodeProtocolTranslatorPB.java:266)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at java.lang.reflect.Method.invoke(Method.java:498)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:258)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at com.sun.proxy.$Proxy11.getBlockLocations(Unknown Source)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:1287)
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - 	... 33 more
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - Process completed unsuccessfully in 1301 seconds.
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import ERROR - Job run failed!
java.lang.RuntimeException: azkaban.jobExecutor.utils.process.ProcessFailureException: Process exited with code 1
	at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:305)
	at azkaban.execapp.JobRunner.runJob(JobRunner.java:787)
	at azkaban.execapp.JobRunner.doRun(JobRunner.java:602)
	at azkaban.execapp.JobRunner.run(JobRunner.java:563)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: azkaban.jobExecutor.utils.process.ProcessFailureException: Process exited with code 1
	at azkaban.jobExecutor.utils.process.AzkabanProcess.run(AzkabanProcess.java:125)
	at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:297)
	... 8 more
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import ERROR - azkaban.jobExecutor.utils.process.ProcessFailureException: Process exited with code 1 cause: azkaban.jobExecutor.utils.process.ProcessFailureException: Process exited with code 1
02-08-2019 10:47:29 CST bi_cal_resume_achieve_sqoop_import INFO - Finishing job bi_cal_resume_achieve_sqoop_import retry: 3 at 1564714049527 with status FAILED

产生这种问题原因是：

第一次拉表，使用了压缩格式：–compression-codec org.apache.hadoop.io.compress.SnappyCodec 
然后这种情况下再次sqoop拉表时，sqoop检测到表已经存在，就不会重新建表，但是我第二次拉表时，使用的命令语句是没有压缩的（sqoop语句中没有上面的参数），存储格式是不一样的，所以导致这个报错，解决办法是删掉这个表，重新导即可，注意以后每次导时要用一样的语句（第一次导时用了压缩，那以后就每次都用压缩的方式）

今天的文章sqoop：File does not exist:分享到此就结束了，感谢您的阅读，如果确实帮到您，您可以动动手指转发给其他人。