tomoyanonymous/quartz-research-note
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs on making plugins

Browse Source
This commit is contained in:
Jacky Zhao 2023-08-10 21:16:07 -07:00
parent 86e41a82de
commit 575d949996
11 changed files with 330 additions and 43 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

287
content/advanced/making plugins.md
View File

@ -2,12 +2,295 @@
title: Making your own plugins
---
This part of the documentation will assume you have some basic coding knowledge in TypeScript and will include code snippets that describe the interface of what Quartz plugins should look like.
> [!warning]
> This part of the documentation will assume you have working knowledge in TypeScript and will include code snippets that describe the interface of what Quartz plugins should look like.
Quartz's plugins are a series of transformations over content. This is illustrated in the diagram of the processing pipeline below:
![[quartz-transform-pipeline.png]]
## Transformers
All plugins are defined as a function that takes in a single parameter for options `type OptionType = object | undefined` and return an object that corresponds to the type of plugin it is.
```ts
type OptionType = object | undefined
type QuartzPlugin<Options extends OptionType = undefined> = (opts?: Options) => QuartzPluginInstance
type QuartzPluginInstance =
| QuartzTransformerPluginInstance
| QuartzFilterPluginInstance
| QuartzEmitterPluginInstance
```
The following sections will go into detail for what methods can be implemented for each plugin type. Before we do that, let's clarify a few more ambiguous types:
- `BuildCtx` is defined in `quartz/ctx.ts`. It consists of
- `argv`: The command line arguments passed to the Quartz [[build]] command
- `cfg`: The full Quartz [[configuration]]
- `allSlugs`: a list of all the valid content slugs (see [[paths]] for more information on what a `ServerSlug` is)
- `StaticResources` is defined in `quartz/resources.tsx`. It consists of
- `css`: a list of URLs for stylesheets that should be loaded
- `js`: a list of scripts that should be loaded. A script is described with the `JSResource` type which is also defined in `quartz/resources.tsx`. It allows you to define a load time (either before or after the DOM has been loaded), whether it should be a module, and either the source URL or the inline content of the script.
## Transformers
Transformers **map** over content, taking a Markdown file and outputting modified content or adding metadata to the file itself.
```ts
export type QuartzTransformerPluginInstance = {
name: string
textTransform?: (ctx: BuildCtx, src: string | Buffer) => string | Buffer
markdownPlugins?: (ctx: BuildCtx) => PluggableList
htmlPlugins?: (ctx: BuildCtx) => PluggableList
externalResources?: (ctx: BuildCtx) => Partial<StaticResources>
}
```
All transformer plugins must define at least a `name` field to register the plugin and a few optional functions that allow you to hook into various parts of transforming a single Markdown file.
- `textTransform` performs a text-to-text transformation *before* a file is parsed into the [Markdown AST](https://github.com/syntax-tree/mdast).
- `markdownPlugins` defines a list of [remark plugins](https://github.com/remarkjs/remark/blob/main/doc/plugins.md). `remark` is a tool that transforms Markdown to Markdown in a structured way.
- `htmlPlugins` defines a list of [rehype plugins](https://github.com/rehypejs/rehype/blob/main/doc/plugins.md). Similar to how `remark` works, `rehype` is a tool that transforms HTML to HTML in a structured way.
- `externalResources` defines any external resources the plugin may need to load on the client-side for it to work properly.
Normally for both `remark` and `rehype`, you can find existing plugins that you can use to . If you'd like to create your own `remark` or `rehype` plugin, checkout the [guide to creating a plugin](https://unifiedjs.com/learn/guide/create-a-plugin/) using `unified` (the underlying AST parser and transformer library).
A good example of a transformer plugin that borrows from the `remark` and `rehype` ecosystems is the [[Latex]] plugin:
```ts title="quartz/plugins/transformers/latex.ts"
import remarkMath from "remark-math"
import rehypeKatex from "rehype-katex"
import rehypeMathjax from "rehype-mathjax/svg.js"
import { QuartzTransformerPlugin } from "../types"
interface Options {
renderEngine: "katex" | "mathjax"
}
export const Latex: QuartzTransformerPlugin<Options> = (opts?: Options) => {
const engine = opts?.renderEngine ?? "katex"
return {
name: "Latex",
markdownPlugins() {
return [remarkMath]
},
htmlPlugins() {
if (engine === "katex") {
// if you need to pass options into a plugin, you
// can use a tuple of [plugin, options]
return [[rehypeKatex, { output: "html" }]]
} else {
return [rehypeMathjax]
}
},
externalResources() {
if (engine === "katex") {
return {
css: [
"https://cdn.jsdelivr.net/npm/katex@0.16.0/dist/katex.min.css",
],
js: [
{
src: "https://cdn.jsdelivr.net/npm/katex@0.16.7/dist/contrib/copy-tex.min.js",
loadTime: "afterDOMReady",
contentType: "external",
},
],
}
} else {
return {}
}
},
}
}
```
Another common thing that transformer plugins will do is parse a file and add extra data for that file:
```ts
export const AddWordCount: QuartzTransformerPlugin = () => {
return {
name: "AddWordCount",
markdownPlugins() {
return [() => {
return (tree, file) => {
// tree is an `mdast` root element
// file is a `vfile`
const text = file.value
const words = text.split(" ").length
file.data.wordcount = words
}
}]
}
}
}
// tell typescript about our custom data fields we are adding
// other plugins will then also be aware of this data field
declare module "vfile" {
interface DataMap {
wordcount: number
}
}
```
Finally, you can also perform transformations over Markdown or HTML ASTs using the `visit` function from the `unist-util-visit` package or the `findAndReplace` function from the `mdast-util-find-and-replace` package.
```ts
export const TextTransforms: QuartzTransformerPlugin = () => {
return {
name: "TextTransforms",
markdownPlugins() {
return [() => {
return (tree, file) => {
// replace _text_ with the italics version
findAndReplace(tree, /_(.+)_/, (_value: string, ...capture: string[]) => {
// inner is the text inside of the () of the regex
const [inner] = capture
// return an mdast node
// https://github.com/syntax-tree/mdast
return {
type: "emphasis",
children: [{ type: 'text', value: inner }]
}
})
// remove all links (replace with just the link content)
// match by 'type' field on an mdast node
// https://github.com/syntax-tree/mdast#link in this example
visit(tree, "link", (link: Link) => {
return {
type: "paragraph"
children: [{ type: 'text', value: link.title }]
}
})
}
}]
}
}
}
```
All transformer plugins can be found under `quartz/plugins/transformers`. If you decide to write your own transformer plugin, don't forget to re-export it under `quartz/plugins/transformers/index.ts`
A parting word: transformer plugins are quite complex so don't worry if you don't get them right away. Take a look at the built in transformers and see how they operate over content to get a better sense for how to accomplish what you are trying to do.
## Filters
Filters **filter** content, taking the output of all the transformers and determining what files to actually keep and what to discard.
```ts
export type QuartzFilterPlugin<Options extends OptionType = undefined> = (
opts?: Options,
) => QuartzFilterPluginInstance
export type QuartzFilterPluginInstance = {
name: string
shouldPublish(ctx: BuildCtx, content: ProcessedContent): boolean
}
```
A filter plugin must define a `name` field and a `shouldPublish` function that takes in a piece of content that has been processed by all the transformers and returns a `true` or `false` depending on whether it should be passed to the emitter plugins or not.
For example, here is the built-in plugin for removing drafts:
```ts title="quartz/plugins/filters/draft.ts"
import { QuartzFilterPlugin } from "../types"
export const RemoveDrafts: QuartzFilterPlugin<{}> = () => ({
name: "RemoveDrafts",
shouldPublish(_ctx, [_tree, vfile]) {
// uses frontmatter parsed from transformers
const draftFlag: boolean = vfile.data?.frontmatter?.draft ?? false
return !draftFlag
},
})
```
## Emitters
Emitters **reduce** over content, taking in a list of all the transformed and filtered content and creating output files.
```ts
export type QuartzEmitterPlugin<Options extends OptionType = undefined> = (
opts?: Options,
) => QuartzEmitterPluginInstance
export type QuartzEmitterPluginInstance = {
name: string
emit(
ctx: BuildCtx,
content: ProcessedContent[],
resources: StaticResources,
emitCallback: EmitCallback,
): Promise<FilePath[]>
getQuartzComponents(ctx: BuildCtx): QuartzComponent[]
}
```
An emitter plugin must define a `name` field an `emit` function and a `getQuartzComponents` function. `emit` is responsible for looking at all the parsed and filtered content and then appropriately creating files and returning a list of paths to files the plugin created.
Creating new files can be done via regular Node [fs module](https://nodejs.org/api/fs.html) (i.e. `fs.cp` or `fs.writeFile`) or via the `emitCallback` if you are creating files that contain text. The `emitCallback` function is the 4th argument of the emit function. It's interface looks something like this:
```ts
export type EmitCallback = (data: {
// the name of the file to emit (not including the file extension)
slug: ServerSlug
// the file extension
ext: `.${string}` | ""
// the file content to add
content: string
}) => Promise<FilePath>
```
This is a thin wrapper around writing to the appropriate output folder and ensuring that intermediate directories exist. If you choose to use the native Node `fs` APIs, ensure you emit to the `argv.output` folder as well.
If you are creating an emitter plugin that needs to render components, there are three more things to be aware of:
- Your component should use `getQuartzComponents` to declare a list of `QuartzComponents` that it uses to construct the page. See the page on [[creating components]] for more information.
- You can use the `renderPage` function defined in `quartz/components/renderPage.tsx` to render Quartz components into HTML.
- If you need to render an HTML AST to JSX, you can use the `toJsxRuntime` function from `hast-util-to-jsx-runtime` library. An example of this can be found in `quartz/components/pages/Content.tsx`.
For example, the following is a simplified version of the content page plugin that renders every single page.
```tsx title="quartz/plugins/emitters/contentPage.tsx"
export const ContentPage: QuartzEmitterPlugin = () => {
// construct the layout
const layout: FullPageLayout = {
...sharedPageComponents,
...defaultContentPageLayout,
pageBody: Content(),
}
const { head, header, beforeBody, pageBody, left, right, footer} = layout
return {
name: "ContentPage",
getQuartzComponents() {
return [head, ...header, ...beforeBody, pageBody, ...left, ...right, footer]
},
async emit(ctx, content, resources, emit): Promise<FilePath[]> {
const cfg = ctx.cfg.configuration
const fps: FilePath[] = []
const allFiles = content.map((c) => c[1].data)
for (const [tree, file] of content) {
const slug = canonicalizeServer(file.data.slug!)
const externalResources = pageResources(slug, resources)
const componentData: QuartzComponentProps = {
fileData: file.data,
externalResources,
cfg,
children: [],
tree,
allFiles,
}
const content = renderPage(slug, componentData, opts, externalResources)
const fp = await emit({
content,
slug: file.data.slug!,
ext: ".html",
})
fps.push(fp)
}
return fps
},
}
}
```
Note that it takes in a `FullPageLayout` as the options. It's made by combining a `SharedLayout` and a `PageLayout` both of which are provided through the `quartz.layout.ts` file.
> [!hint]
> Look in `quartz/plugins` for more examples of plugins in Quartz as reference for your own plugins!

8
content/advanced/paths.md
View File

@ -43,3 +43,11 @@ graph LR
Server --"canonicalizeServer()"--> Canonical
style Canonical stroke-width:4px
```
Here are the main types of slugs with a rough description of each type of path:
- `ClientSlug`: client-side slug, usually obtained through `window.location`. Contains the protocol (i.e. starts with `https://`)
- `CanonicalSlug`: should be used whenever you need to refer to the location of a file/note. Shouldn't be a relative path and shouldn't have leading or trailing slashes `/` either. Also shouldn't have `/index` as an ending or a file extension.
- `RelativeURL`: must start with `.` or `..` to indicate it's a relative URL. Shouldn't have `/index` as an ending or a file extension.
- `ServerSlug`: cannot be relative and may not have leading or trailing slashes.
- `FilePath`: a real file path to a file on disk. Cannot be relative and must have a file extension.
To get a clearer picture of how these relate to each other, take a look at the path tests in `quartz/path.test.ts`.

6
content/configuration.md
View File

@ -58,9 +58,9 @@ plugins: {
}
```
- [[making plugins#Transformers|Transformers]] **map** over content, taking a Markdown file and outputting modified content or adding metadata to the file itself (e.g. parsing frontmatter, generating a description)
- [[making plugins#Filters|Filters]] **filter** content, taking the output of all the transformers and determining what files to actually keep and what to discord (e.g. filtering out drafts)
- [[making plugins#Emitters|Emitters]] **reduce** over content, taking in a list of all the transformed and filtered content and creating output files (e.g. creating an RSS feed or pages that list all files with a specific tag)
- [[making plugins#Transformers|Transformers]] **map** over content (e.g. parsing frontmatter, generating a description)
- [[making plugins#Filters|Filters]] **filter** content (e.g. filtering out drafts)
- [[making plugins#Emitters|Emitters]] **reduce** over content (e.g. creating an RSS feed or pages that list all files with a specific tag)
By adding, removing, and reordering plugins from the `tranformers`, `filters`, and `emitters` fields, you can customize the behaviour of Quartz.

1
content/features/upcoming features.md
View File

@ -6,6 +6,7 @@ draft: true
- docs for making plugins
- nested tags showing duplicate
- tag page markdown file for description not being rendered
- back button with anchors / popovers + spa is broken
- search should be fast for large repos
- debounce cfg rebuild on large repos

3
quartz/plugins/emitters/aliases.ts
View File

@ -30,9 +30,8 @@ export const AliasRedirects: QuartzEmitterPlugin = () => ({
for (const alias of aliases) {
const slug = path.posix.join(dir, alias) as ServerSlug
const fp = (slug + ".html") as FilePath
const redirUrl = resolveRelative(canonicalizeServer(slug), ogSlug)
await emit({
const fp = await emit({
content: `
<!DOCTYPE html>
<html lang="en-us">

1
quartz/plugins/emitters/componentResources.ts
View File

@ -98,7 +98,6 @@ function addGlobalPageResources(
componentResources.afterDOMLoaded.push(plausibleScript)
}
// spa
if (cfg.enableSPA) {
componentResources.afterDOMLoaded.push(spaRouterScript)
} else {

15
quartz/plugins/emitters/contentIndex.ts
View File

@ -88,21 +88,19 @@ export const ContentIndex: QuartzEmitterPlugin<Partial<Options>> = (opts) => {
}
if (opts?.enableSiteMap) {
await emit({
emitted.push(await emit({
content: generateSiteMap(cfg, linkIndex),
slug: "sitemap" as ServerSlug,
ext: ".xml",
})
emitted.push("sitemap.xml" as FilePath)
}))
}
if (opts?.enableRSS) {
await emit({
emitted.push(await emit({
content: generateRSSFeed(cfg, linkIndex),
slug: "index" as ServerSlug,
ext: ".xml",
})
emitted.push("index.xml" as FilePath)
}))
}
const fp = path.join("static", "contentIndex") as ServerSlug
@ -117,12 +115,11 @@ export const ContentIndex: QuartzEmitterPlugin<Partial<Options>> = (opts) => {
}),
)
await emit({
emitted.push(await emit({
content: JSON.stringify(simplifiedIndex),
slug: fp,
ext: ".json",
})
emitted.push(`${fp}.json` as FilePath)
}))
return emitted
},

4
quartz/plugins/emitters/contentPage.tsx
View File

@ -42,9 +42,7 @@ export const ContentPage: QuartzEmitterPlugin<Partial<FullPageLayout>> = (userOp
}
const content = renderPage(slug, componentData, opts, externalResources)
const fp = (file.data.slug + ".html") as FilePath
await emit({
const fp = await emit({
content,
slug: file.data.slug!,
ext: ".html",

4
quartz/plugins/emitters/folderPage.tsx
View File

@ -74,9 +74,7 @@ export const FolderPage: QuartzEmitterPlugin<FullPageLayout> = (userOpts) => {
}
const content = renderPage(slug, componentData, opts, externalResources)
const fp = (file.data.slug! + ".html") as FilePath
await emit({
const fp = await emit({
content,
slug: file.data.slug!,
ext: ".html",

4
quartz/plugins/emitters/tagPage.tsx
View File

@ -80,9 +80,7 @@ export const TagPage: QuartzEmitterPlugin<FullPageLayout> = (userOpts) => {
}
const content = renderPage(slug, componentData, opts, externalResources)
const fp = (file.data.slug + ".html") as FilePath
await emit({
const fp = await emit({
content,
slug: file.data.slug!,
ext: ".html",

40
quartz/plugins/transformers/latex.ts
View File

@ -15,25 +15,31 @@ export const Latex: QuartzTransformerPlugin<Options> = (opts?: Options) => {
return [remarkMath]
},
htmlPlugins() {
return [engine === "katex" ? [rehypeKatex, { output: "html" }] : [rehypeMathjax]]
if (engine === "katex") {
return [[rehypeKatex, { output: "html" }]]
} else {
return [rehypeMathjax]
}
},
externalResources() {
return engine === "katex"
? {
css: [
// base css
"https://cdn.jsdelivr.net/npm/katex@0.16.0/dist/katex.min.css",
],
js: [
{
// fix copy behaviour: https://github.com/KaTeX/KaTeX/blob/main/contrib/copy-tex/README.md
src: "https://cdn.jsdelivr.net/npm/katex@0.16.7/dist/contrib/copy-tex.min.js",
loadTime: "afterDOMReady",
contentType: "external",
},
],
}
: {}
if (engine === "katex") {
return {
css: [
// base css
"https://cdn.jsdelivr.net/npm/katex@0.16.0/dist/katex.min.css",
],
js: [
{
// fix copy behaviour: https://github.com/KaTeX/KaTeX/blob/main/contrib/copy-tex/README.md
src: "https://cdn.jsdelivr.net/npm/katex@0.16.7/dist/contrib/copy-tex.min.js",
loadTime: "afterDOMReady",
contentType: "external",
},
],
}
} else {
return {}
}
},
}
}