How to suppress printing of variable values in zeppelin

Given the following snippet:

val data = sc.parallelize(0 until 10000)
val local = data.collect 

Zeppelin prints out the entire value of local to the notebook screen. How may that behavior be changed?

Since 0.6.0, Zeppelin provides a boolean flag zeppelin.spark.printREPLOutput in spark's interpreter configuration (accessible via the GUI), which is set to true by default. If you set its value to false then you get the desired behaviour that only explicit print statements are output.

See also:

How to suppress printing of variable values in zeppelin, Given the following snippet: val data = sc.parallelize(0 until 10000) val local = data.collect println(s"local.size") Zeppelin prints out the entire value of local to the​  How to suppress printing of variable values in zeppelin Given the following snippet: val data=sc.parallelize(0 until 10000) val local=data.collect println(s"local.size") Zeppelin prints out the entire value of local to the notebook screen.

You can also try adding curly brackets around your code.

{val data = sc.parallelize(0 until 10000)
val local = data.collect 

Re: Is there a way to show only form output in a Zeppelin paragraph?, I allow the user to select values from a dropdown, and want to show them the picklist, but not show them the Scala output underneath it. If I use the "Hide Output"  Hi, Is there a way to suppress the output of statements? For example, could I suppress "x: Int = 1" below? In [24]: val x = 1 x: Int = 1 Thanks, Praveen

FWIW, this appears to be new behaviour. Until recently we have been using Livy 0.4, it only output the content of the final statement (rather than echoing the output of the whole script).

When we upgraded to Livy 0.5, the behaviour changed to output the entire script.

While splitting the paragraph and hiding the output does work, it seems like an unnecessary overhead to the usability of Zeppelin. for example, if you need to refresh your output, then you have to remember to run two paragraphs (i.e. the one that sets up your output and the one containing the actual println).

There are, IMHO, other usability issues with this approach that makes, again IMHO, Zeppelin less intuitive to use.

Someone has logged this JIRA ticket to address "the problem", please vote for it: LIVY-507

[#ZEPPELIN-688] Giving an option to hide REPL output in Spark , ZEPPELIN-688. Giving an option to hide REPL output in Spark interpreter. Status: Assignee: that users can hide this. ( Default value is as-is )  The variable will be substituted by a comma-separated string based on the selected items. For example: You can specify the delimiter using ${checkbox(delimiter):formName=}: Like select form, the paragraph will be automatically run after you change your selection by default. But in case you have multiple types dynamic form in one paragraph, you might want to run the paragraph after changing all the selections.

Zeppelin, as well as spark-shell REPL, always prints the whole interpreter output.

If you really want to have only local.size string printed - best way to do it is to put println "local.size" statement inside the separate paragraph.

Then you can hide all output of the previous paragraph using small "book" icon on the top-right.

Apache Zeppelin 0.8.2 Documentation: Interpreter in Apache Zeppelin, Properties are exported as environment variables when property name is of interpreter by add #{contextParameterName} in value, parameter can be of the are removed from Notebook output to prevent accidentally leaking passwords. When you use () in a foreach call you may only pass a simple expression as argument. To make your argument a simple expression you have to wrap threshold:Double also in (). score_threshold.foreach ( (threshold: Double) => { val counts = DF.filter ($"score" >= threshold) .groupBy ().agg (

What I do to avoid this is define a top-level function, and then call it:

def run() : Unit = {
    val data = sc.parallelize(0 until 10000)
    val local = data.collect 

zeppelin Archives, If you wish to display the data on a map and in a graph using the same Then a watcher is made to see when this variable changes so the new value can be You edit the code in the web interface but it can hide it once you have the form  Apache Spark is a fast and general-purpose cluster computing system. It provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution graphs. Apache Spark is supported in Zeppelin with Spark interpreter group which consists of below five interpreters. SparkInterpreter.

zeppelin/interpreter-setting.json at master · apache/zeppelin · GitHub, zeppelin/spark/interpreter/src/main/resources/interpreter-setting.json "​description": "Max number of Spark SQL result to display.", In Kubernetes mode​, value can be Jinja template string with 3 template variables 'PORT', '​SERVICE_NAME' and 'SERVICE_DOMAIN'. "description": "Whether hide spark ui in zeppelin ui",. The following example uses the NOPRINT option to suppress output to any ODS destination. The OUTP= option creates an output data set that contains the correlation coefficients between all numerical variables in the Sashelp.Cars data for each of the 38 values of the categorical variable Make:

Zeppelin/Spark/Cassandra integration tutorial, But making Zeppelin supporting the Spark-Cassandra integration requires some The value should point to a single or a list of IP addresses of your Cassandra cluster the Spark-Cassandra dependencies to the SPARK_CLASSPATH variable. Logging.scala[logInfo]:58) – Got job 0 (reduce at :35) with 2 output partitions Usage Note 22393: Can I suppress the printing of repeated values within my PROC PRINT output? If you specify an identical ID and BY statement on the variables where you want to suppress repeated values, PROC PRINT will produce such a specially formatted report.

Dev Bytes, QueryBuilder UI in Apache Zeppelin. In the last post we QueryBuilder plugin. First we setup a few variables in Spark. //Get list of distinct values on a column for given table //Wrapper for printing any DataFrame in Zeppelin table format <​input id="whereClause" type="text" ng-model="clause" class="hide"></input>. To set a default value for a text input form, use the ${variable_name=value} syntax. Note that traditionally in SQL queries text values in WHERE clauses are enclosed with single quotes (e.g. WHERE column = 'value'). However, surrounding a variable/template with single quotes can cause issues with the Zeppelin interpreter.

  • Apparently the zeppelin developers finally woke up to this. But I am in general v unhappy with usability of zeppelin and switched to jupyter
  • For me it's exactly the other way round. I'm leaving jupyter because it separates too strongly between different kernels and between back- and frontend. Zeppelin not only allows you to mix python with scala / spark, etc., plus reactive AngularJS-elements, but also to inject via print("%html " + ...) arbitrary html/javascript (e.g., D3, Plotly, etc.) directly into the frontend. Btw: I compile 0.7.0 manually to already get some improvements. However, interactive usability is still one step behind jupyter, and jupyter-lab looks very promising for the future, too...
  • k Your comments about not intermixing scala / python are true for jupyter. I really appreciate the high end keyboarding and generally good user experience in jupyter: zeppelin is way behind to the point of being unusable to me.
  • How can the same be achieved in Jupyter Notebook with Scala kernel?
  • Ok that's a nice solution
  • Note that this solution also changes the scope of your val data and local.
  • @Paul-ArmandVerhaegen thanks for pointing that out. I think the same goes for the solution that was accepted by the OP.
  • @Caner Indeed, the accepted answer also changes the scope of the vars, but I commented on your solution since the scope change is more obvious in the function approach, and your answer has more votes (including mine). The reason I posted this comment, is that after implementing the proposed solutions, the new local vars shadow the global ones and that can be tricky when the spark interpreter is not restarted to clear the global vars (in Zeppelin).
  • "as well as spark-shell REPL". No - the REPL does not. in the spark-shell the statement "val local=data.collect" will not result in any printing statements.
  • @javadba May be I misunderstood you somehow, in ./bin/spark-shell here is the output I get scala>val local = data.collect local: Array[Int] = Array(0, 1, 2, 3, ...