about云开发

 找回密码
 立即注册

QQ登录

只需一步,快速开始

扫一扫,访问微社区

查看: 1829|回复: 3

Spark local -kafk求救

[复制链接]

2

主题

1

听众

0

收听

新手上路

Rank: 1

积分
38
QQ
发表于 2018-1-8 10:09:49 | 显示全部楼层 |阅读模式
在Spark 程序中怎们能消除kafka自带的log4j的INFO的信息?????,网上查的很多都试过了
现在就是想在Spark local程序中消除INFO信息,不想打印INFO的信息?求救??????
发表于 2018-1-8 10:20:10 | 显示全部楼层

可以把日志级别设置为warn或则error。

下是如何设置warn的
spark所有应用,可以Java工程目录中新建/src/main/resources目录,把log4j.properties放置该目录。
1.png
log4j.properties生成:
1. Spark中conf默认配置文件是log4j.properties.template,可以将其改名为log4j.properties;
2. 将Spark-core包中的log4j-default.properties内容复制到log4j.properties文件。
#log4j内容如下

[XML] 纯文本查看 复制代码
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    [url]http://www.apache.org/licenses/LICENSE-2.0[/url]
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# Set everything to be logged to the console
log4j.rootCategory=WARN, console
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.err
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n

# Settings to quiet third party logs that are too verbose
log4j.logger.org.spark-project.jetty=WARN
log4j.logger.org.spark-project.jetty.util.component.AbstractLifeCycle=ERROR
log4j.logger.org.apache.spark.repl.SparkIMain$exprTyper=INFO
log4j.logger.org.apache.spark.repl.SparkILoop$SparkILoopInterpreter=INFO
log4j.logger.org.apache.parquet=ERROR
log4j.logger.parquet=ERROR

# SPARK-9183: Settings to avoid annoying messages when looking up nonexistent UDFs in SparkSQL with Hive support
log4j.logger.org.apache.hadoop.hive.metastore.RetryingHMSHandler=FATAL
log4j.logger.org.apache.hadoop.hive.ql.exec.FunctionRegistry=ERROR


在开发工程中,我们可以设置日志级别为WARN,即:
log4j.rootCategory=WARN, console


2

主题

1

听众

0

收听

新手上路

Rank: 1

积分
38
QQ
 楼主| 发表于 2018-1-8 10:25:32 | 显示全部楼层
行,谢了,我先试试

10

主题

4

听众

0

收听

中级会员

Rank: 3Rank: 3

积分
277
发表于 2018-1-26 11:29:30 | 显示全部楼层
设置日志级别Logger.getLogger("org").setLevel(Level.ERROR)只有报错时才输出
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

站长推荐上一条 /1 下一条

QQ|小黑屋|about云开发-学问论坛|社区 ( 京ICP备12023829号

GMT+8, 2018-10-20 16:34 , Processed in 0.377459 second(s), 33 queries , Gzip On.

Powered by Discuz! X3.2 Licensed

快速回复 返回顶部 返回列表