Question - How to override log levels?

[sanket-t-shah]

Here is something that I see very heavily on console. Is there any way to remove INFO logs for DotnetWorker, SimpleWorker and TaskRunner classes? Tried via log4j.properties, but it didn't helped.

DotnetWorker PID:[29968] Args:[-m pyspark.worker] SparkVersion:[2.4.5]
[2020-05-17T07:23:33.4117588Z] [SANKETSHAH-HP] [Info] [SimpleWorker] RunSimpleWorker() is starting with port = 24058.
[2020-05-17T07:23:33.4500182Z] [SANKETSHAH-HP] [Info] [TaskRunner] [0] Starting with ReuseSocket[False].
[2020-05-17T07:23:33.5350621Z] [SANKETSHAH-HP] [Debug] [TaskRunner] [0] Received END_OF_STREAM signal.
[2020-05-17T07:23:33.5353116Z] [SANKETSHAH-HP] [Info] [TaskRunner] [0] Processed a task: readComplete:True, entries:0
[2020-05-17T07:23:33.5354370Z] [SANKETSHAH-HP] [Info] [TaskRunner] [0] Waiting for JVM side to close socket.
[2020-05-17T07:23:33.5388400Z] [SANKETSHAH-HP] [Info] [TaskRunner] [0] JVM side has closed socket.
[2020-05-17T07:23:33.5395995Z] [SANKETSHAH-HP] [Info] [TaskRunner] [0] Finished running 1 task(s).
[2020-05-17T07:23:33.5396603Z] [SANKETSHAH-HP] [Info] [SimpleWorker] RunSimpleWorker() finished successfully
DotnetWorker PID:[17044] Args:[-m pyspark.worker] SparkVersion:[2.4.5]
[2020-05-17T07:23:33.7544054Z] [SANKETSHAH-HP] [Info] [SimpleWorker] RunSimpleWorker() is starting with port = 24060.
[2020-05-17T07:23:33.7940088Z] [SANKETSHAH-HP] [Info] [TaskRunner] [0] Starting with ReuseSocket[False].
[2020-05-17T07:23:33.8654407Z] [SANKETSHAH-HP] [Debug] [TaskRunner] [0] Received END_OF_STREAM signal.
[2020-05-17T07:23:33.8656834Z] [SANKETSHAH-HP] [Info] [TaskRunner] [0] Processed a task: readComplete:True, entries:0
[2020-05-17T07:23:33.8658278Z] [SANKETSHAH-HP] [Info] [TaskRunner] [0] Waiting for JVM side to close socket.
[2020-05-17T07:23:33.8662172Z] [SANKETSHAH-HP] [Info] [TaskRunner] [0] JVM side has closed socket.
[2020-05-17T07:23:33.8668915Z] [SANKETSHAH-HP] [Info] [TaskRunner] [0] Finished running 1 task(s).
[2020-05-17T07:23:33.8669161Z] [SANKETSHAH-HP] [Info] [SimpleWorker] RunSimpleWorker() finished successfully
DotnetWorker PID:[24696] Args:[-m pyspark.worker] SparkVersion:[2.4.5]
[2020-05-17T07:23:37.3720401Z] [SANKETSHAH-HP] [Info] [SimpleWorker] RunSimpleWorker() is starting with port = 24069.

[imback82]

Currently, you cannot turn this off and it is being tracked by #473.