Doris消费Kafka数据

1、查看load task

show routine Load;  

SHOW  ROUTINE LOAD\G;

2、暂停一个load Task

PAUSE  ROUTINE LOAD FOR [job_name];               


PAUSE ROUTINE LOAD FOR load_test_user_3;

3、删除一个 

STOP ROUTINE LOAD FOR load_test_user_4;

4、查看日志，一定要查看be.INFO 

I0430 14:24:36.568722  6899 data_consumer.h:88] kafka error: Local: Host resolution failure, event: uat-datacenter2:9092/100: Failed to resolve 'uat-datacenter2:9092': Name or service not known (after 514035102ms in state INIT)
I0430 14:24:36.568774  6899 data_consumer.h:88] kafka error: Local: Host resolution failure, event: uat-datacenter1:9092/98: Failed to resolve 'uat-datacenter1:9092': Name or service not known (after 514035102ms in state INIT)
I0430 14:24:36.579171  6899 data_consumer.h:88] kafka error: Local: Host resolution failure, event: uat-datacenter3:9092/99: Failed to resolve 'uat-datacenter3:9092': Name or service not known (after 514035112ms in state INIT)

表示不知道kafka地址

5、错误信息

ReasonOfStateChanged: ErrorReason{code=errCode = 4, msg='Job failed to fetch all current partition with error errCode = 2, detailMessage = Failed to get all partitions of kafka topic: doris_test. error: errCode = 2, detailMessage = failed to get kafka partition info: [failed to get partition meta: Local: Broker transport failure]'}

表示分区信息获取不到

6、查看routine 建立信息

help routine load

7、查看建表信息

help create  table

 

#############案例##################

1、创建一个用户表

CREATE TABLE user
(
id INT DEFAULT '10',
name VARCHAR(32) DEFAULT '',
age DECIMAL(10, 2) DEFAULT "10.5",
create_time datetime NULL COMMENT ""
)
ENGINE=olap
UNIQUE KEY(id)
DISTRIBUTED BY HASH(id) BUCKETS 10
PROPERTIES("replication_num" = "1");

2、创建一个routine load

create routine load load_test_user_1 on user
columns (id,name,age,create_time),
columns terminated by ","
PROPERTIES (
    "desired_concurrent_number"="1",  
    "max_error_number"="1000"
)
FROM KAFKA (
    "kafka_broker_list"= "uat-datacenter2:9092",
    "kafka_topic" = "doris_test"

);

3、查看表信息

select * from user

　　

标签：load,消费,uat,9092,kafka,user,error,Kafka,Doris
来源： https://www.cnblogs.com/ywjfx/p/14722302.html