zeppelin-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From prabhjyotsi...@apache.org
Subject [zeppelin] branch master updated: [ZEPPELIN-4311] Supporting new parser for markdown as pegdown parser …
Date Mon, 23 Sep 2019 04:45:24 GMT
This is an automated email from the ASF dual-hosted git repository.

prabhjyotsingh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git


The following commit(s) were added to refs/heads/master by this push:
     new 0d9239e  [ZEPPELIN-4311] Supporting new parser for markdown as pegdown parser …
0d9239e is described below

commit 0d9239ef333e04e0788dbccfd1d00314b97a4370
Author: Bhavik Patel <bhavikpatel552@gmail.com>
AuthorDate: Tue Sep 17 12:30:47 2019 +0530

    [ZEPPELIN-4311] Supporting new parser for markdown as pegdown parser …
    
    ### What is this PR for?
    Supporting new parser flexmark for markdown, as pegdown parser is deprecated and facing
issue with pegdown parser.
    
    ### What type of PR is it?
    [Bug Fix | Improvement]
    
    ### What is the Jira issue?
    https://issues.apache.org/jira/browse/ZEPPELIN-4311
    
    ### How should this be tested?
    CI Pass
    
    ### Screenshots (if appropriate)
    
    ### Questions:
    * Does the licenses files need update? No
    * Is there breaking changes for older versions? No
    * Does this needs documentation? No
    
    Author: Bhavik Patel <bhavikpatel552@gmail.com>
    
    Closes #3443 from bhavikpatel9977/ZEPPELIN-4311 and squashes the following commits:
    
    59edd8d73 [Bhavik Patel] Incroporated the review changes
    0f5980381 [Bhavik Patel] added license and documentation
    e28f68226 [Bhavik Patel] Added license
    e4b54ce4a [Bhavik Patel] [ZEPPELIN-4311] Supporting new parser for markdown as pegdown
parser is deprecated and facing issue with parser
    
    Change-Id: I4aa3bc93267b3a5d5003af176f84aa37e4c742ff
---
 LICENSE                                            |   2 +
 ...markdown-example-flexmark-parser-extensions.png | Bin 0 -> 41115 bytes
 .../docs-img/markdown-example-flexmark-parser.png  | Bin 0 -> 31019 bytes
 docs/interpreter/markdown.md                       |  20 +-
 markdown/README.md                                 |  34 +++
 markdown/pom.xml                                   |   7 +
 .../apache/zeppelin/markdown/FlexmarkParser.java   |  73 +++++++
 .../org/apache/zeppelin/markdown/Markdown.java     |  12 +-
 .../apache/zeppelin/markdown/UMLBlockQuote.java    | 118 +++++++++++
 .../zeppelin/markdown/UMLBlockQuoteParser.java     | 187 +++++++++++++++++
 .../org/apache/zeppelin/markdown/UMLExtension.java |  57 +++++
 .../apache/zeppelin/markdown/UMLNodeRenderer.java  | 231 +++++++++++++++++++++
 .../src/main/resources/interpreter-setting.json    |   4 +-
 .../zeppelin/markdown/FlexmarkParserTest.java      | 225 ++++++++++++++++++++
 14 files changed, 959 insertions(+), 11 deletions(-)

diff --git a/LICENSE b/LICENSE
index 40c1143..121d2e1 100644
--- a/LICENSE
+++ b/LICENSE
@@ -287,6 +287,8 @@ The following components are provided under the BSD 2-Clause license.
 See file
   (BSD 2 Clause) portions of SQLLine (http://sqlline.sourceforge.net/) - http://sqlline.sourceforge.net/#license
    jdbc/src/main/java/org/apache/zeppelin/jdbc/SqlCompleter.java
 
+  (BSD 2-clause) flexmark-java (https://github.com/vsch/flexmark-java)
+
 ========================================================================
 Jython Software License
 ========================================================================
diff --git a/docs/assets/themes/zeppelin/img/docs-img/markdown-example-flexmark-parser-extensions.png
b/docs/assets/themes/zeppelin/img/docs-img/markdown-example-flexmark-parser-extensions.png
new file mode 100644
index 0000000..8af5c9e
Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/markdown-example-flexmark-parser-extensions.png
differ
diff --git a/docs/assets/themes/zeppelin/img/docs-img/markdown-example-flexmark-parser.png
b/docs/assets/themes/zeppelin/img/docs-img/markdown-example-flexmark-parser.png
new file mode 100644
index 0000000..87f9faa
Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/markdown-example-flexmark-parser.png
differ
diff --git a/docs/interpreter/markdown.md b/docs/interpreter/markdown.md
index 609f204..e06c563 100644
--- a/docs/interpreter/markdown.md
+++ b/docs/interpreter/markdown.md
@@ -25,7 +25,7 @@ limitations under the License.
 
 ## Overview
 [Markdown](http://daringfireball.net/projects/markdown/) is a plain text formatting syntax
designed so that it can be converted to HTML.
-Apache Zeppelin uses [pegdown](https://github.com/sirthias/pegdown) and [markdown4j](https://github.com/jdcasey/markdown4j)
as markdown parsers.
+Apache Zeppelin uses [flexmark](https://github.com/vsch/flexmark-java), [pegdown](https://github.com/sirthias/pegdown)
and [markdown4j](https://github.com/jdcasey/markdown4j) as markdown parsers.
 
 In Zeppelin notebook, you can use ` %md ` in the beginning of a paragraph to invoke the Markdown
interpreter and generate static html from Markdown plain text.
 
@@ -53,21 +53,25 @@ For more information, please see [Mathematical Expression](../usage/display_syst
   </tr>
   <tr>
     <td>markdown.parser.type</td>
-    <td>pegdown</td>
-    <td>Markdown Parser Type. <br/> Available values: pegdown, markdown4j.</td>
+    <td>flexmark</td>
+    <td>Markdown Parser Type. <br/> Available values: flexmark, pegdown, markdown4j.</td>
   </tr>
 </table>
 
+### Flexmark parser (Default Markdown Parser)
 
-### Pegdown Parser
+CommonMark/Markdown Java parser with source level AST.
+
+<img src="{{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/markdown-example-flexmark-parser.png"
width="70%" />
 
-`pegdown` parser provides github flavored markdown.
+`flexmark` parser provides [YUML](http://yuml.me/) and [Websequence](https://www.websequencediagrams.com/)
extensions also.
 
-<img src="{{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/markdown-example-pegdown-parser.png"
width="70%" />
+<img src="{{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/markdown-example-flexmark-parser-extensions.png"
width="70%" />
 
-`pegdown` parser provides [YUML](http://yuml.me/) and [Websequence](https://www.websequencediagrams.com/)
plugins also. 
+### Pegdown Parser
 
-<img src="{{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/markdown-example-pegdown-parser-plugins.png"
width="70%" />
+`pegdown` parser provides github flavored markdown. Although still one of the most popular
Markdown parsing libraries for the JVM, pegdown has reached its end of life.
+The project is essentially unmaintained with tickets piling up and crucial bugs not being
fixed.`pegdown`'s parsing performance isn't great. But keep this parser for the backward compatibility.
 
 ### Markdown4j Parser
 
diff --git a/markdown/README.md b/markdown/README.md
new file mode 100644
index 0000000..24f5dce
--- /dev/null
+++ b/markdown/README.md
@@ -0,0 +1,34 @@
+# Overview
+Markdown parsers for Apache Zeppelin. Markdown is a plain text formatting syntax designed
so that it can be converted to HTML. Apache Zeppelin uses `flexmark`, `pegdown` and `markdown4j`.
+Since both `pegdown` and `markdown4j` are deprecated but it support for backward compatibility.
+
+# Architecture
+Current interpreter implementation creates the instance of parser based on the configuration
parameter provided, default is `flexmark` through `Markdown` and render the text into html.
+
+### Technical overview
+When interpreter is starting it check which parser has configured in  `markdown.parser.type`,
based on this value, it creates the instance of that parser. For each subsequent request,
+respective parser get the markdown text, parses it and renders into the html.
+
+### Flexmark Parser Overview
+
+CommonMark/Markdown Java parser with source level AST.
+
+### Flexmark Requirement
+
+ * maven dependency to add in pom.xml
+ 
+ ```
+<flexmark.all.version>0.50.40</flexmark.all.version>
+
+ <dependency>
+       <groupId>com.vladsch.flexmark</groupId>
+       <artifactId>flexmark-all</artifactId>
+       <version>${flexmark.all.version}</version>
+ </dependency>
+ ```
+
+### Flexmark Technical overview for Custom Extension
+To support, YUML and websequnce diagram, need to build the image URL from the respective
block and render it into HTML, So it requires 
+to implement some custom classes. `UMLExtension` is base class which has factory for other
classes like `UMLBlockQuoteParser` and `UMLNodeRenderer`. 
+`UMLBlockQuoteParser` which parses the UML block and creates block quote node `UMLBlockQuote`.

+`UMLNodeRenderer` which builds the URL using this block quote node `UMLBlockQuote` and render
it as image into HTML. 
\ No newline at end of file
diff --git a/markdown/pom.xml b/markdown/pom.xml
index 7d6bcff..7720353 100644
--- a/markdown/pom.xml
+++ b/markdown/pom.xml
@@ -38,6 +38,7 @@
     <commons.lang3.version>3.4</commons.lang3.version>
     <markdown4j.version>2.2-cj-1.0</markdown4j.version>
     <pegdown.version>1.6.0</pegdown.version>
+    <flexmark.all.version>0.50.40</flexmark.all.version>
   </properties>
 
   <dependencies>
@@ -55,6 +56,12 @@
     </dependency>
 
     <dependency>
+       <groupId>com.vladsch.flexmark</groupId>
+       <artifactId>flexmark-all</artifactId>
+       <version>${flexmark.all.version}</version>
+    </dependency>
+
+    <dependency>
       <groupId>org.apache.commons</groupId>
       <artifactId>commons-lang3</artifactId>
       <version>${commons.lang3.version}</version>
diff --git a/markdown/src/main/java/org/apache/zeppelin/markdown/FlexmarkParser.java b/markdown/src/main/java/org/apache/zeppelin/markdown/FlexmarkParser.java
new file mode 100644
index 0000000..f4de944
--- /dev/null
+++ b/markdown/src/main/java/org/apache/zeppelin/markdown/FlexmarkParser.java
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zeppelin.markdown;
+
+
+import com.vladsch.flexmark.ext.autolink.AutolinkExtension;
+import com.vladsch.flexmark.ext.gfm.strikethrough.StrikethroughExtension;
+import com.vladsch.flexmark.ext.tables.TablesExtension;
+import com.vladsch.flexmark.ext.typographic.TypographicExtension;
+import com.vladsch.flexmark.ext.wikilink.WikiLinkExtension;
+import com.vladsch.flexmark.util.ast.Node;
+import com.vladsch.flexmark.html.HtmlRenderer;
+import com.vladsch.flexmark.parser.Parser;
+import com.vladsch.flexmark.util.data.MutableDataSet;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.Arrays;
+
+/**
+ * Flexmark Parser
+ */
+public class FlexmarkParser implements MarkdownParser {
+  private static final Logger LOGGER = LoggerFactory.getLogger(FlexmarkParser.class);
+  private Parser parser;
+  private HtmlRenderer renderer;
+
+  public FlexmarkParser() {
+    MutableDataSet options = new MutableDataSet();
+    options.set(Parser.EXTENSIONS, Arrays.asList(StrikethroughExtension.create(),
+        TablesExtension.create(),
+        UMLExtension.create(),
+        AutolinkExtension.create(),
+        WikiLinkExtension.create(),
+        TypographicExtension.create()));
+    options.set(HtmlRenderer.SOFT_BREAK, "<br />\n");
+    parser = Parser.builder(options).build();
+    renderer = HtmlRenderer.builder(options).build();
+  }
+
+  @Override
+  public String render(String markdownText) {
+    Node document = parser.parse(markdownText);
+    String html = renderer.render(document);
+    return wrapWithMarkdownClassDiv(html);
+  }
+
+  /**
+   * wrap with markdown class div to styling DOM using css.
+   */
+  public static String wrapWithMarkdownClassDiv(String html) {
+    return new StringBuilder()
+        .append("<div class=\"markdown-body\">\n")
+        .append(html)
+        .append("\n</div>")
+        .toString();
+  }
+}
diff --git a/markdown/src/main/java/org/apache/zeppelin/markdown/Markdown.java b/markdown/src/main/java/org/apache/zeppelin/markdown/Markdown.java
index 83b4069..99aaac2 100644
--- a/markdown/src/main/java/org/apache/zeppelin/markdown/Markdown.java
+++ b/markdown/src/main/java/org/apache/zeppelin/markdown/Markdown.java
@@ -19,7 +19,6 @@ package org.apache.zeppelin.markdown;
 
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
-
 import java.util.List;
 import java.util.Properties;
 
@@ -56,12 +55,21 @@ public class Markdown extends Interpreter {
       public String toString() {
         return PARSER_TYPE_MARKDOWN4J;
       }
+    },
+
+    FLEXMARK {
+      @Override
+      public String toString() {
+        return PARSER_TYPE_FLEXMARK;
+      }
     }
+
   }
 
   public static final String MARKDOWN_PARSER_TYPE = "markdown.parser.type";
   public static final String PARSER_TYPE_PEGDOWN = "pegdown";
   public static final String PARSER_TYPE_MARKDOWN4J = "markdown4j";
+  public static final String PARSER_TYPE_FLEXMARK = "flexmark";
 
   public Markdown(Properties property) {
     super(property);
@@ -72,6 +80,8 @@ public class Markdown extends Interpreter {
 
     if (MarkdownParserType.PEGDOWN.toString().equals(parserType)) {
       return new PegdownParser();
+    } else if (MarkdownParserType.FLEXMARK.toString().equals(parserType)) {
+      return new FlexmarkParser();
     } else {
       // default parser
       return new Markdown4jParser();
diff --git a/markdown/src/main/java/org/apache/zeppelin/markdown/UMLBlockQuote.java b/markdown/src/main/java/org/apache/zeppelin/markdown/UMLBlockQuote.java
new file mode 100644
index 0000000..4b07134
--- /dev/null
+++ b/markdown/src/main/java/org/apache/zeppelin/markdown/UMLBlockQuote.java
@@ -0,0 +1,118 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zeppelin.markdown;
+
+import com.vladsch.flexmark.ast.Paragraph;
+import com.vladsch.flexmark.ast.ParagraphContainer;
+import com.vladsch.flexmark.util.ast.Block;
+import com.vladsch.flexmark.util.ast.BlockContent;
+import com.vladsch.flexmark.util.sequence.BasedSequence;
+
+import java.util.List;
+
+/**
+ * Block quote which acts as a node.
+ */
+public class UMLBlockQuote extends Block implements ParagraphContainer {
+
+  private BasedSequence openingMarker = BasedSequence.NULL;
+  private BasedSequence openingTrailing = BasedSequence.NULL;
+  private BasedSequence closingMarker = BasedSequence.NULL;
+  private BasedSequence closingTrailing = BasedSequence.NULL;
+
+  @Override
+  public void getAstExtra(StringBuilder out) {
+    segmentSpanChars(out, openingMarker, "open");
+    segmentSpanChars(out, openingTrailing, "openTrail");
+    segmentSpanChars(out, closingMarker, "close");
+    segmentSpanChars(out, closingTrailing, "closeTrail");
+  }
+
+  @Override
+  public BasedSequence[] getSegments() {
+    return new BasedSequence[]{openingMarker, openingTrailing, closingMarker, closingTrailing};
+  }
+
+  @Override
+  public boolean isParagraphEndWrappingDisabled(final Paragraph node) {
+    return node == getLastChild() || node.getNext() instanceof UMLBlockQuote;
+  }
+
+  @Override
+  public boolean isParagraphStartWrappingDisabled(final Paragraph node) {
+    return node == getFirstChild() || node.getPrevious() instanceof UMLBlockQuote;
+  }
+
+  public UMLBlockQuote() {
+  }
+
+  public UMLBlockQuote(BasedSequence chars) {
+    super(chars);
+  }
+
+  public UMLBlockQuote(BasedSequence chars, List<BasedSequence> segments) {
+    super(chars, segments);
+  }
+
+  public UMLBlockQuote(BlockContent blockContent) {
+    super(blockContent);
+  }
+
+  public BasedSequence getOpeningMarker() {
+    return openingMarker;
+  }
+
+  public void setOpeningMarker(BasedSequence openingMarker) {
+    this.openingMarker = openingMarker;
+  }
+
+  public BasedSequence getClosingMarker() {
+    return closingMarker;
+  }
+
+  public void setClosingMarker(final BasedSequence closingMarker) {
+    this.closingMarker = closingMarker;
+  }
+
+  public BasedSequence getOpeningTrailing() {
+    return openingTrailing;
+  }
+
+  public void setOpeningTrailing(final BasedSequence openingTrailing) {
+    this.openingTrailing = openingTrailing;
+  }
+
+  public BasedSequence getClosingTrailing() {
+    return closingTrailing;
+  }
+
+  public void setClosingTrailing(final BasedSequence closingTrailing) {
+    this.closingTrailing = closingTrailing;
+  }
+
+  @Override
+  public String toString() {
+    return "YUMLBlockQuote{" +
+        "openingMarker=" + openingMarker +
+        ", openingTrailing=" + openingTrailing +
+        ", closingMarker=" + closingMarker +
+        ", closingTrailing=" + closingTrailing +
+        ", lineSegments=" + lineSegments +
+        '}';
+  }
+}
diff --git a/markdown/src/main/java/org/apache/zeppelin/markdown/UMLBlockQuoteParser.java
b/markdown/src/main/java/org/apache/zeppelin/markdown/UMLBlockQuoteParser.java
new file mode 100644
index 0000000..9d56856
--- /dev/null
+++ b/markdown/src/main/java/org/apache/zeppelin/markdown/UMLBlockQuoteParser.java
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zeppelin.markdown;
+
+import com.vladsch.flexmark.ext.gitlab.internal.GitLabOptions;
+import com.vladsch.flexmark.parser.InlineParser;
+import com.vladsch.flexmark.parser.block.AbstractBlockParser;
+import com.vladsch.flexmark.parser.block.BlockContinue;
+import com.vladsch.flexmark.parser.block.BlockParser;
+import com.vladsch.flexmark.parser.block.BlockParserFactory;
+import com.vladsch.flexmark.parser.block.AbstractBlockParserFactory;
+import com.vladsch.flexmark.parser.block.BlockStart;
+import com.vladsch.flexmark.parser.block.MatchedBlockParser;
+import com.vladsch.flexmark.parser.block.ParserState;
+import com.vladsch.flexmark.parser.block.CustomBlockParserFactory;
+import com.vladsch.flexmark.util.ast.Block;
+import com.vladsch.flexmark.util.ast.BlockContent;
+import com.vladsch.flexmark.util.ast.Node;
+import com.vladsch.flexmark.util.data.DataHolder;
+import com.vladsch.flexmark.util.sequence.BasedSequence;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.List;
+import java.util.Set;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+/**
+ * Parser to get the Block content
+ */
+public class UMLBlockQuoteParser extends AbstractBlockParser {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(UMLBlockQuoteParser.class);
+
+
+  private static Pattern YUML_BLOCK_START = Pattern.compile("(%%%)\\s+(.*\\n)");
+  private static Pattern YUML_BLOCK_END = Pattern.compile("%%%(\\s*$)");
+
+  private final UMLBlockQuote block = new UMLBlockQuote();
+  private BlockContent content = new BlockContent();
+  private final GitLabOptions options;
+  private boolean hadClose = false;
+
+  UMLBlockQuoteParser(DataHolder options, BasedSequence openMarker, BasedSequence openTrailing)
{
+    this.options = new GitLabOptions(options);
+    this.block.setOpeningMarker(openMarker);
+    this.block.setOpeningTrailing(openTrailing);
+  }
+
+  @Override
+  public Block getBlock() {
+    return block;
+  }
+
+  @Override
+  public BlockContinue tryContinue(ParserState state) {
+    if (hadClose) {
+      return BlockContinue.none();
+    }
+
+    final int index = state.getIndex();
+
+    BasedSequence line = state.getLineWithEOL();
+    final Matcher matcher = YUML_BLOCK_END.matcher(line.subSequence(index));
+    if (!matcher.matches()) {
+      return BlockContinue.atIndex(index);
+    } else {
+      // if have open gitlab block quote last child then let them handle it
+      Node lastChild = block.getLastChild();
+      if (lastChild instanceof UMLBlockQuote) {
+        final BlockParser parser = state.getActiveBlockParser((Block) lastChild);
+        if (parser instanceof UMLBlockQuoteParser && !((UMLBlockQuoteParser) parser).hadClose)
{
+          // let the child handle it
+          return BlockContinue.atIndex(index);
+        }
+      }
+      hadClose = true;
+      block.setClosingMarker(state.getLine().subSequence(index, index + 3));
+      block.setClosingTrailing(state.getLineWithEOL().subSequence(matcher.start(1),
+          matcher.end(1)));
+      return BlockContinue.atIndex(state.getLineEndIndex());
+    }
+  }
+
+  @Override
+  public void addLine(ParserState state, BasedSequence line) {
+    content.add(line, state.getIndent());
+  }
+
+  @Override
+  public void closeBlock(ParserState state) {
+    block.setContent(content);
+    block.setCharsFromContent();
+    content = null;
+  }
+
+  @Override
+  public boolean isContainer() {
+    return true;
+  }
+
+  @Override
+  public boolean canContain(final ParserState state, final BlockParser blockParser,
+                            final Block block) {
+    return true;
+  }
+
+  @Override
+  public void parseInlines(InlineParser inlineParser) {
+  }
+
+  /**
+   * Generic Factory
+   */
+  public static class Factory implements CustomBlockParserFactory {
+    @Override
+    public Set<Class<? extends CustomBlockParserFactory>> getAfterDependents()
{
+      return null;
+    }
+
+    @Override
+    public Set<Class<? extends CustomBlockParserFactory>> getBeforeDependents()
{
+      return null;
+    }
+
+    @Override
+    public boolean affectsGlobalScope() {
+      return false;
+    }
+
+    @Override
+    public BlockParserFactory apply(DataHolder options) {
+      return new UMLBlockQuoteParser.BlockFactory(options);
+    }
+  }
+
+  private static class BlockFactory extends AbstractBlockParserFactory {
+    private final GitLabOptions options;
+
+    BlockFactory(DataHolder options) {
+      super(options);
+      this.options = new GitLabOptions(options);
+    }
+
+    boolean haveBlockQuoteParser(ParserState state) {
+      final List<BlockParser> parsers = state.getActiveBlockParsers();
+      int i = parsers.size();
+      while (i-- > 0) {
+        if (parsers.get(i) instanceof UMLBlockQuoteParser) {
+          return true;
+        }
+      }
+      return false;
+    }
+
+    @Override
+    public BlockStart tryStart(ParserState state, MatchedBlockParser matchedBlockParser)
{
+      if (options.nestedBlockQuotes || !haveBlockQuoteParser(state)) {
+        BasedSequence line = state.getLineWithEOL();
+        final Matcher matcher = YUML_BLOCK_START.matcher(line);
+        if (matcher.matches()) {
+          LOGGER.debug("Matcher group count {} ", matcher.groupCount());
+          return BlockStart.of(new UMLBlockQuoteParser(state.getProperties(),
+              line.subSequence(0, 3), line.subSequence(4, line.length())))
+              .atIndex(state.getLineEndIndex());
+        }
+      }
+      return BlockStart.none();
+    }
+  }
+}
diff --git a/markdown/src/main/java/org/apache/zeppelin/markdown/UMLExtension.java b/markdown/src/main/java/org/apache/zeppelin/markdown/UMLExtension.java
new file mode 100644
index 0000000..4956a31
--- /dev/null
+++ b/markdown/src/main/java/org/apache/zeppelin/markdown/UMLExtension.java
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zeppelin.markdown;
+
+import com.vladsch.flexmark.html.HtmlRenderer;
+import com.vladsch.flexmark.parser.Parser;
+import com.vladsch.flexmark.util.data.MutableDataHolder;
+import com.vladsch.flexmark.util.builder.Extension;
+
+
+/**
+ * Extension to support YUML and web sequnce diagram.
+ */
+public class UMLExtension implements Parser.ParserExtension, HtmlRenderer.HtmlRendererExtension
{
+
+  private UMLExtension() {
+
+  }
+
+  public static Extension create() {
+    return new UMLExtension();
+  }
+
+  @Override
+  public void rendererOptions(MutableDataHolder options) {
+
+  }
+
+  @Override
+  public void extend(HtmlRenderer.Builder rendererBuilder, String rendererType) {
+    rendererBuilder.nodeRendererFactory(new UMLNodeRenderer.Factory());
+  }
+
+  @Override
+  public void parserOptions(MutableDataHolder options) {
+  }
+
+  @Override
+  public void extend(Parser.Builder parserBuilder) {
+    parserBuilder.customBlockParserFactory(new UMLBlockQuoteParser.Factory());
+  }
+}
diff --git a/markdown/src/main/java/org/apache/zeppelin/markdown/UMLNodeRenderer.java b/markdown/src/main/java/org/apache/zeppelin/markdown/UMLNodeRenderer.java
new file mode 100644
index 0000000..39406f9
--- /dev/null
+++ b/markdown/src/main/java/org/apache/zeppelin/markdown/UMLNodeRenderer.java
@@ -0,0 +1,231 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zeppelin.markdown;
+
+import com.vladsch.flexmark.ext.gitlab.internal.GitLabOptions;
+import com.vladsch.flexmark.html.CustomNodeRenderer;
+import com.vladsch.flexmark.html.HtmlWriter;
+import com.vladsch.flexmark.html.renderer.NodeRenderer;
+import com.vladsch.flexmark.html.renderer.NodeRendererFactory;
+import com.vladsch.flexmark.html.renderer.NodeRendererContext;
+import com.vladsch.flexmark.html.renderer.NodeRenderingHandler;
+import com.vladsch.flexmark.util.data.DataHolder;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.io.OutputStreamWriter;
+import java.io.UnsupportedEncodingException;
+import java.net.URL;
+import java.net.URLConnection;
+import java.net.URLEncoder;
+import java.nio.charset.StandardCharsets;
+
+import java.util.Map;
+import java.util.Set;
+import java.util.HashSet;
+import java.util.HashMap;
+import java.util.Objects;
+
+import static org.apache.commons.lang3.StringUtils.defaultString;
+
+/**
+ * Html Node renderer to render the image
+ */
+public class UMLNodeRenderer implements NodeRenderer {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(UMLNodeRenderer.class);
+
+  public static final String YUML = "yuml";
+  public static final String SEQUENCE = "sequence";
+  public static final String WEBSEQ_URL = "http://www.websequencediagrams.com";
+
+  final GitLabOptions options;
+
+  public UMLNodeRenderer(DataHolder options) {
+    this.options = new GitLabOptions(options);
+  }
+
+  @Override
+  public Set<NodeRenderingHandler<?>> getNodeRenderingHandlers() {
+    Set<NodeRenderingHandler<?>> set = new HashSet<>();
+    set.add(new NodeRenderingHandler<>(UMLBlockQuote.class,
+        new CustomNodeRenderer<UMLBlockQuote>() {
+          @Override
+          public void render(UMLBlockQuote node, NodeRendererContext context,
+                             HtmlWriter html) {
+            UMLNodeRenderer.this.render(node, context, html);
+          }
+        }));
+    return set;
+  }
+
+
+  private void render(final UMLBlockQuote node, final NodeRendererContext context,
+                      HtmlWriter html) {
+    LOGGER.debug("Rendering HTML");
+
+    String firstLine = node.getOpeningTrailing().toString();
+    String[] splitWithSpace = firstLine.split(" ");
+
+    LOGGER.debug("Start of the node {} ", firstLine);
+    LOGGER.debug("Content within block {} ", node.getFirstChild().getChars());
+
+    Map<String, String> paramMap = new HashMap<>();
+    for (int i = 1; i < splitWithSpace.length; i++) {
+      String[] splitWithEqual = splitWithSpace[i].split("=");
+      paramMap.put(splitWithEqual[0], splitWithEqual[1]);
+    }
+
+    String url = "";
+
+    if (splitWithSpace[0].equals(YUML) && !Objects.isNull(node.getFirstChild()))
{
+      url = createYumlUrl(paramMap, node.getFirstChild().getChars().toString());
+      LOGGER.debug("Encoded YUML URL {} ", url);
+    } else if (splitWithSpace[0].equals(SEQUENCE) && !Objects.isNull(node.getFirstChild()))
{
+      url = createWebsequenceUrl(paramMap.get("style"), node.getFirstChild().getChars().toString());
+      LOGGER.debug("Encoded web sequence diagram URL {} ", url);
+    } else {
+      html.withAttr().tagLineIndent("blockquote", new Runnable() {
+        @Override
+        public void run() {
+          context.renderChildren(node);
+        }
+      });
+      return;
+    }
+
+    html.attr("src", url);
+    html.attr("alt", "");
+    html.srcPos(node.getChars()).withAttr().tagVoid("img");
+  }
+
+  /**
+   * Factory for node renderer
+   */
+  public static class Factory implements NodeRendererFactory {
+    @Override
+    public NodeRenderer apply(final DataHolder options) {
+      return new UMLNodeRenderer(options);
+    }
+  }
+
+  public static String createYumlUrl(Map<String, String> params, String body) {
+    StringBuilder inlined = new StringBuilder();
+    for (String line : body.split("\\r?\\n")) {
+      line = line.trim();
+      if (line.length() > 0) {
+        if (inlined.length() > 0) {
+          inlined.append(", ");
+        }
+        inlined.append(line);
+      }
+    }
+
+    String encodedBody = null;
+    try {
+      encodedBody = URLEncoder.encode(inlined.toString(), "UTF-8");
+    } catch (UnsupportedEncodingException e) {
+      new RuntimeException("Failed to encode YUML markdown body", e);
+    }
+
+    StringBuilder mergedStyle = new StringBuilder();
+    String style = defaultString(params.get("style"), "scruffy");
+    String type = defaultString(params.get("type"), "class");
+    String format = defaultString(params.get("format"), "svg");
+
+    mergedStyle.append(style);
+
+    if (null != params.get("dir")) {
+      mergedStyle.append(";dir:" + params.get("dir"));
+    }
+
+    if (null != params.get("scale")) {
+      mergedStyle.append(";scale:" + params.get("scale"));
+    }
+
+    return new StringBuilder()
+        .append("http://yuml.me/diagram/")
+        .append(mergedStyle.toString() + "/")
+        .append(type + "/")
+        .append(encodedBody)
+        .append("." + format)
+        .toString();
+  }
+
+  public static String createWebsequenceUrl(String style,
+                                            String content) {
+
+    style = StringUtils.defaultString(style, "default");
+
+    OutputStreamWriter writer = null;
+    BufferedReader reader = null;
+
+    String webSeqUrl = "";
+
+    try {
+      String query = new StringBuilder()
+          .append("style=")
+          .append(style)
+          .append("&message=")
+          .append(URLEncoder.encode(content, "UTF-8"))
+          .append("&apiVersion=1")
+          .toString();
+
+      URL url = new URL(WEBSEQ_URL);
+      URLConnection conn = url.openConnection();
+      conn.setDoOutput(true);
+      writer = new OutputStreamWriter(conn.getOutputStream(), StandardCharsets.UTF_8);
+      writer.write(query);
+      writer.flush();
+
+      StringBuilder response = new StringBuilder();
+      reader = new BufferedReader(
+          new InputStreamReader(conn.getInputStream(), StandardCharsets.UTF_8));
+      String line;
+      while ((line = reader.readLine()) != null) {
+        response.append(line);
+      }
+
+      writer.close();
+      reader.close();
+
+      String json = response.toString();
+
+      int start = json.indexOf("?png=");
+      int end = json.indexOf("\"", start);
+
+      if (start != -1 && end != -1) {
+        webSeqUrl = WEBSEQ_URL + "/" + json.substring(start, end);
+        System.out.println("websequrl : " + webSeqUrl);
+      }
+    } catch (IOException e) {
+      throw new RuntimeException("Failed to get proper response from websequencediagrams.com",
e);
+    } finally {
+      IOUtils.closeQuietly(writer);
+      IOUtils.closeQuietly(reader);
+    }
+
+    return webSeqUrl;
+  }
+}
diff --git a/markdown/src/main/resources/interpreter-setting.json b/markdown/src/main/resources/interpreter-setting.json
index d2a59c8..da4b59b 100644
--- a/markdown/src/main/resources/interpreter-setting.json
+++ b/markdown/src/main/resources/interpreter-setting.json
@@ -7,8 +7,8 @@
       "markdown.parser.type": {
         "envName": "MARKDOWN_PARSER_TYPE",
         "propertyName": "markdown.parser.type",
-        "defaultValue": "pegdown",
-        "description": "Markdown Parser Type. Available values: pegdown, markdown4j. Default
= pegdown",
+        "defaultValue": "flexmark",
+        "description": "Markdown Parser Type. Available values: pegdown, markdown4j, flexmark.
Default = flexmark",
         "type": "string"
       }
     },
diff --git a/markdown/src/test/java/org/apache/zeppelin/markdown/FlexmarkParserTest.java b/markdown/src/test/java/org/apache/zeppelin/markdown/FlexmarkParserTest.java
new file mode 100644
index 0000000..a80337e
--- /dev/null
+++ b/markdown/src/test/java/org/apache/zeppelin/markdown/FlexmarkParserTest.java
@@ -0,0 +1,225 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zeppelin.markdown;
+
+import org.apache.zeppelin.interpreter.InterpreterResult;
+import org.hamcrest.CoreMatchers;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.ErrorCollector;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.ArrayList;
+import java.util.Properties;
+
+import static org.apache.zeppelin.markdown.FlexmarkParser.wrapWithMarkdownClassDiv;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertThat;
+
+public class FlexmarkParserTest {
+
+  Logger logger = LoggerFactory.getLogger(FlexmarkParserTest.class);
+  Markdown md;
+
+  @Rule
+  public ErrorCollector collector = new ErrorCollector();
+
+  @Before
+  public void setUp() throws Exception {
+    Properties props = new Properties();
+    props.put(Markdown.MARKDOWN_PARSER_TYPE, Markdown.PARSER_TYPE_FLEXMARK);
+    md = new Markdown(props);
+    md.open();
+  }
+
+  @After
+  public void tearDown() throws Exception {
+    md.close();
+  }
+
+  @Test
+  public void testMultipleThread() {
+    ArrayList<Thread> arrThreads = new ArrayList<Thread>();
+    for (int i = 0; i < 10; i++) {
+      Thread t = new Thread() {
+        public void run() {
+          String r1 = null;
+          try {
+            r1 = md.interpret("# H1", null).code().name();
+          } catch (Exception e) {
+            logger.error("testTestMultipleThread failed to interpret", e);
+          }
+          collector.checkThat("SUCCESS",
+              CoreMatchers.containsString(r1));
+        }
+      };
+      t.start();
+      arrThreads.add(t);
+    }
+
+    for (int i = 0; i < 10; i++) {
+      try {
+        arrThreads.get(i).join();
+      } catch (InterruptedException e) {
+        logger.error("testTestMultipleThread failed to join threads", e);
+      }
+    }
+  }
+
+  @Test
+  public void testStrikethrough() {
+    InterpreterResult result = md.interpret("This is ~~deleted~~ text", null);
+    assertEquals(wrapWithMarkdownClassDiv("<p>This is <del>deleted</del>
text</p>\n"),
+        result.message().get(0).getData());
+  }
+
+  @Test
+  public void testHeader() {
+    InterpreterResult r1 = md.interpret("# H1", null);
+    assertEquals(wrapWithMarkdownClassDiv("<h1>H1</h1>\n"), r1.message().get(0).getData());
+
+    InterpreterResult r2 = md.interpret("## H2", null);
+    assertEquals(wrapWithMarkdownClassDiv("<h2>H2</h2>\n"), r2.message().get(0).getData());
+
+    InterpreterResult r3 = md.interpret("### H3", null);
+    assertEquals(wrapWithMarkdownClassDiv("<h3>H3</h3>\n"), r3.message().get(0).getData());
+
+    InterpreterResult r4 = md.interpret("#### H4", null);
+    assertEquals(wrapWithMarkdownClassDiv("<h4>H4</h4>\n"), r4.message().get(0).getData());
+
+    InterpreterResult r5 = md.interpret("##### H5", null);
+    assertEquals(wrapWithMarkdownClassDiv("<h5>H5</h5>\n"), r5.message().get(0).getData());
+
+    InterpreterResult r6 = md.interpret("###### H6", null);
+    assertEquals(wrapWithMarkdownClassDiv("<h6>H6</h6>\n"), r6.message().get(0).getData());
+
+    InterpreterResult r7 = md.interpret("Alt-H1\n" + "======", null);
+    assertEquals(wrapWithMarkdownClassDiv("<h1>Alt-H1</h1>\n"), r7.message().get(0).getData());
+
+    InterpreterResult r8 = md.interpret("Alt-H2\n" + "------", null);
+    assertEquals(wrapWithMarkdownClassDiv("<h2>Alt-H2</h2>\n"), r8.message().get(0).getData());
+  }
+
+  @Test
+  public void testItalics() {
+    InterpreterResult result = md.interpret("This is *italics* text", null);
+
+    assertEquals(
+        wrapWithMarkdownClassDiv("<p>This is <em>italics</em> text</p>\n"),
+        result.message().get(0).getData());
+  }
+
+  @Test
+  public void testStrongEmphasis() {
+    InterpreterResult result = md.interpret("This is **strong emphasis** text", null);
+    assertEquals(
+        wrapWithMarkdownClassDiv("<p>This is <strong>strong emphasis</strong>
text</p>\n"),
+        result.message().get(0).getData());
+  }
+
+  @Test
+  public void testOrderedList() {
+    String input =
+        new StringBuilder()
+            .append("1. First ordered list item\n")
+            .append("2. Another item")
+            .toString();
+
+    String expected =
+        new StringBuilder()
+            .append("<ol>\n")
+            .append("<li>First ordered list item</li>\n")
+            .append("<li>Another item</li>\n")
+            .append("</ol>\n")
+            .toString();
+
+    InterpreterResult result = md.interpret(input, null);
+
+
+    assertEquals(wrapWithMarkdownClassDiv(expected), result.message().get(0).getData());
+  }
+
+  @Test
+  public void testUnorderedList() {
+    String input =
+        new StringBuilder()
+            .append("* Unordered list can use asterisks\n")
+            .append("- Or minuses\n")
+            .append("+ Or pluses")
+            .toString();
+
+    String expected =
+        new StringBuilder()
+            .append("<ul>\n")
+            .append("<li>Unordered list can use asterisks</li>\n")
+            .append("</ul>\n")
+            .append("<ul>\n")
+            .append("<li>Or minuses</li>\n")
+            .append("</ul>\n")
+            .append("<ul>\n")
+            .append("<li>Or pluses</li>\n")
+            .append("</ul>\n")
+            .toString();
+
+    InterpreterResult result = md.interpret(input, null);
+
+    assertEquals(wrapWithMarkdownClassDiv(expected), result.message().get(0).getData());
+  }
+
+  @Test
+  public void testYumlPlugin() {
+    String input = new StringBuilder()
+        .append("%%% yuml style=nofunky scale=120 format=svg\n")
+        .append("[Customer]<>-orders>[Order]\n")
+        .append("[Order]++-0..>[LineItem]\n")
+        .append("[Order]-[note:Aggregate root.]\n")
+        .append("  %%%  ")
+        .toString();
+
+    InterpreterResult result = md.interpret(input, null);
+    assertThat(result.message().get(0).getData(), CoreMatchers
+        .containsString("<img src=\"http://yuml.me/diagram/"));
+  }
+
+  @Test
+  public void testWebsequencePlugin() {
+    String input =
+        new StringBuilder()
+            .append("%%% sequence style=modern-blue\n")
+            .append("title Authentication Sequence\n")
+            .append("Alice->Bob: Authentication Request\n")
+            .append("note right of Bob: Bob thinks about it\n")
+            .append("Bob->Alice: Authentication Response\n")
+            .append("  %%%  ")
+            .toString();
+
+    InterpreterResult result = md.interpret(input, null);
+    
+    System.err.println(result.message().get(0).getData());
+    if (!result.message().get(0).getData().contains(
+        "<img src=\"http://www.websequencediagrams.com/?png=")) {
+      logger.error("Expected {} but found {}",
+          "<img src=\"http://www.websequencediagrams.com/?png=", result.message().get(0).getData());
+    }
+  }
+
+}
+


Mime
View raw message