Skip to content

How html2md is semi auto generated

Jump to bottom
suntong edited this page Aug 24, 2020 · 19 revisions

CLI definition semi-auto generation

Options definition semi-auto generation

Copy from https://pkg.go.dev/github.com/JohannesKaufmann/html-to-markdown?tab=doc#Options, the part of:

type Options struct {
	// "setext" or "atx"
	// default: "atx"
	HeadingStyle string

	// Any Thematic break
	// default: "* * *"
	HorizontalRule string

	// "-", "+", or "*"
	// default: "-"
	BulletListMarker string

	// "indented" or "fenced"
	// default: "indented"
	CodeBlockStyle string

	// ``` or ~~~
	// default: ```
	Fence string

	// _ or *
	// default: _
	EmDelimiter string

	// ** or __
	// default: **
	StrongDelimiter string

	// inlined or referenced
	// default: inlined
	LinkStyle string

	// full, collapsed, or shortcut
	// default: full
	LinkReferenceStyle string
	// contains filtered or unexported fields
}

Then

xclip -o | sed '/ string/!d;' | sed 's/^\t//; s/ string//; s/^/ - /; 1s/^/Options:\n/' | tee /tmp/jkmd-Options.yaml

will get:

Options:
 - HeadingStyle
 - HorizontalRule
 - BulletListMarker
 - CodeBlockStyle
 - Fence
 - EmDelimiter
 - StrongDelimiter
 - LinkStyle
 - LinkReferenceStyle

Prepare /tmp/jkmd-Options.tmpl as:

{{range .Options}}
  - Name: Opt{{.}}
    Type: "string"
    Flag: "opt-{{clc2ss . | css2lk}}"
    Usage: Option {{.}}
{{end}}

Run

easygen /tmp/jkmd-Options

will yield:

  - Name: OptHeadingStyle
    Type: "string"
    Flag: "opt-heading-style"
    Usage: Option HeadingStyle

  - Name: OptHorizontalRule
    Type: "string"
    Flag: "opt-horizontal-rule"
    Usage: Option HorizontalRule

  - Name: OptBulletListMarker
    Type: "string"
    Flag: "opt-bullet-list-marker"
    Usage: Option BulletListMarker

  - Name: OptCodeBlockStyle
    Type: "string"
    Flag: "opt-code-block-style"
    Usage: Option CodeBlockStyle

  - Name: OptFence
    Type: "string"
    Flag: "opt-fence"
    Usage: Option Fence

  - Name: OptEmDelimiter
    Type: "string"
    Flag: "opt-em-delimiter"
    Usage: Option EmDelimiter

  - Name: OptStrongDelimiter
    Type: "string"
    Flag: "opt-strong-delimiter"
    Usage: Option StrongDelimiter

  - Name: OptLinkStyle
    Type: "string"
    Flag: "opt-link-style"
    Usage: Option LinkStyle

  - Name: OptLinkReferenceStyle
    Type: "string"
    Flag: "opt-link-reference-style"
    Usage: Option LinkReferenceStyle

Plugins definition semi-auto generation

Version 1

Just combine the results together from plain shell commands. E.g.,

$ ls $GOPATH/src/github.com/JohannesKaufmann/html-to-markdown/plugin/ | sed 's/.go$//; /_test$/d; s/_/-/g; s/^/plugin-/; s/-block$//; s/confluence-/conf-/; s/^/    Flag: "/; s/$/"/'
    Flag: "plugin-conf-attachment"
    Flag: "plugin-conf-code"
    Flag: "plugin-frontmatter"
    Flag: "plugin-gfm"
    Flag: "plugin-strikethrough"
    Flag: "plugin-table"
    Flag: "plugin-task-list"
    Flag: "plugin-vimeo"
    Flag: "plugin-youtube"

Because it does not involve easygen so the details are omitted here.

Version 2

UPDATE

After html-to-markdown has a new plugin which is embedded in the same table.go file, I'm taking a new approach:

First, prepare /tmp/jkmd-Plugins.tmpl as:

{{range .Plugins}}
  - Name: Plugin{{.}}
    Type: bool
    Flag: "plugin-{{clc2ss . | css2lk}}"
    Usage: Plugin {{.}}
{{end}}

Then

$ grep '^func.* md.Plugin ' $GOPATH/src/github.com/JohannesKaufmann/html-to-markdown/plugin/* -h | sed 's/^func/ -/; s/(.*$//; 1s/^/Plugins:\n/;' | tee /tmp/jkmd-Plugins.yaml
Plugins:
 - ConfluenceAttachments
 - ConfluenceCodeBlock
 - EXPERIMENTALFrontMatter
 - GitHubFlavored
 - Strikethrough
 - TableCompat
 - Table
 - TaskListItems
 - EXPERIMENTALVimeoEmbed

Then do easygen /tmp/jkmd-Plugins which produces:

  - Name: PluginConfluenceAttachments
    Type: bool
    Flag: "plugin-confluence-attachments"
    Usage: Plugin ConfluenceAttachments

  - Name: PluginConfluenceCodeBlock
    Type: bool
    Flag: "plugin-confluence-code-block"
    Usage: Plugin ConfluenceCodeBlock

  - Name: PluginEXPERIMENTALFrontMatter
    Type: bool
    Flag: "plugin-experimental-front-matter"
    Usage: Plugin EXPERIMENTALFrontMatter

  - Name: PluginGitHubFlavored
    Type: bool
    Flag: "plugin-git-hub-flavored"
    Usage: Plugin GitHubFlavored

  - Name: PluginStrikethrough
    Type: bool
    Flag: "plugin-strikethrough"
    Usage: Plugin Strikethrough

  - Name: PluginTableCompat
    Type: bool
    Flag: "plugin-table-compat"
    Usage: Plugin TableCompat

  - Name: PluginTable
    Type: bool
    Flag: "plugin-table"
    Usage: Plugin Table

  - Name: PluginTaskListItems
    Type: bool
    Flag: "plugin-task-list-items"
    Usage: Plugin TaskListItems

  - Name: PluginEXPERIMENTALVimeoEmbed
    Type: bool
    Flag: "plugin-experimental-vimeo-embed"
    Usage: Plugin EXPERIMENTALVimeoEmbed

Then do further minor changes either by further scripting or tweaking manually, the details are omitted.

Code

Options handling

Prepare /tmp/html2md-Options.tmpl as:

{{range .Options}}
	if rootArgv.{{.Name}} != "" {
		opt.{{substr .Name 3}} = rootArgv.{{.Name}}
	}{{end}}

Then

  • do easygen /tmp/html2md-Options.tmpl html2md_cli.yaml | xclip -i
  • then paste the result into your editor
  • pick the following segment out from the results and paste it into the .go code
	if rootArgv.OptHeadingStyle != "" {
		opt.HeadingStyle = rootArgv.OptHeadingStyle
	}
	if rootArgv.OptHorizontalRule != "" {
		opt.HorizontalRule = rootArgv.OptHorizontalRule
	}
	if rootArgv.OptBulletListMarker != "" {
		opt.BulletListMarker = rootArgv.OptBulletListMarker
	}
	if rootArgv.OptCodeBlockStyle != "" {
		opt.CodeBlockStyle = rootArgv.OptCodeBlockStyle
	}
	if rootArgv.OptFence != "" {
		opt.Fence = rootArgv.OptFence
	}
	if rootArgv.OptEmDelimiter != "" {
		opt.EmDelimiter = rootArgv.OptEmDelimiter
	}
	if rootArgv.OptStrongDelimiter != "" {
		opt.StrongDelimiter = rootArgv.OptStrongDelimiter
	}
	if rootArgv.OptLinkStyle != "" {
		opt.LinkStyle = rootArgv.OptLinkStyle
	}
	if rootArgv.OptLinkReferenceStyle != "" {
		opt.LinkReferenceStyle = rootArgv.OptLinkReferenceStyle
	}

Case 2

If you still have /tmp/jkmd-Options.yaml around, then prepare /tmp/html2md-Options0.tmpl as:

{{range .Options}}
	if rootArgv.Opt{{.}} != "" {
		opt.{{.}} = rootArgv.Opt{{.}}
	}{{end}}

do easygen /tmp/html2md-Options0.tmpl /tmp/jkmd-Options.yaml will get the same result as above as well.

Plugins handling

Prepare /tmp/html2md-Plugins-cli.tmpl as:

{{range .Options}}{{if gt (.Name | len) 6}}
	if rootArgv.{{.Name}} {
		conv.Use(plugin.{{substr .Name 6}}())
	}{{end}}{{end}}

Then

  • do easygen /tmp/html2md-Plugins-cli.tmpl html2md_cli.yaml | xclip -i
  • then paste the result into your editor
  • pick the following segment out from the results and paste it into the .go code
        if rootArgv.PluginConfluenceAttachments {
                conv.Use(plugin.ConfluenceAttachments())
        }
        if rootArgv.PluginConfluenceCodeBlock {
                conv.Use(plugin.ConfluenceCodeBlock())
        }
        if rootArgv.PluginFrontMatter {
                conv.Use(plugin.FrontMatter())
        }
        if rootArgv.PluginGitHubFlavored {
                conv.Use(plugin.GitHubFlavored())
        }
        if rootArgv.PluginStrikethrough {
                conv.Use(plugin.Strikethrough())
        }
        if rootArgv.PluginTable {
                conv.Use(plugin.Table())
        }
        if rootArgv.PluginTaskListItems {
                conv.Use(plugin.TaskListItems())
        }
        if rootArgv.PluginVimeoEmbed {
                conv.Use(plugin.VimeoEmbed())
        }
        if rootArgv.PluginYoutubeEmbed {
                conv.Use(plugin.YoutubeEmbed())
        }

Then do adjustments (commenting out) as needed.

Take-away

  • The easygen is a simple tool that will make things easier for us, but not doing everything for us.
  • Think of it as a power tool --
    • As a nail-gun, it can save us from hammering nails ourselves, but we still need to hold, point, and trigger it ourselves.
    • As a polishing tool, it still need us to provide & change the sand-pads ourselves.
  • I.e., it takes a way the pain of mechanical labor, and we as the human being need to finish the last mile, most often the logic part (e.g., where to put down the nails, and when to change the sand-pads etc)
Clone this wiki locally