PkgTemplates Developer Guide

Issues and pull requests are welcome! New contributors should make sure to read the ColPrac Contributor Guide.

PkgTemplates can be easily extended by adding new Plugins.

There are three types of plugins: Plugin, FilePlugin, and BadgePlugin.

PkgTemplates.PluginType

Plugins are PkgTemplates' source of customization and extensibility. Add plugins to your Templates to enable extra pieces of repository setup.

When implementing a new plugin, subtype this type to have full control over its behaviour.

source

Template + Package Creation Pipeline

The Template constructor basically does this:

- extract values from keyword arguments
- create a Template from the values
- for each plugin:
  - validate plugin against the template

The plugin validation step uses the validate function. It lets us catch mistakes before we try to generate packages.

PkgTemplates.validateFunction
validate(::Plugin, ::Template)

Perform any required validation for a Plugin.

It is preferred to do validation here instead of in prehook, because this function is called at Template construction time, whereas the prehook is only run at package generation time.

source

The package generation process looks like this:

- create empty directory for the package
- for each plugin, ordered by priority:
  - run plugin prehook
- for each plugin, ordered by priority:
  - run plugin hook
- for each plugin, ordered by priority:
  - run plugin posthook

As you can tell, plugins play a central role in setting up a package.

The three main entrypoints for plugins to do work are the prehook, the hook, and the posthook. As the names might imply, they basically mean "before the main stage", "the main stage", and "after the main stage", respectively.

Each stage is basically identical, since the functions take the exact same arguments. However, the multiple stages allow us to depend on artifacts of the previous stages. For example, the Git plugin uses posthook to commit all generated files, but it wouldn't make sense to do that before the files are generated.

But what about dependencies within the same stage? In this case, we have priority to define which plugins go when. The Git plugin also uses this function to lower its posthook's priority, so that even if other plugins generate files in their posthooks, they still get committed (provided that those plugins didn't set an even lower priority).

PkgTemplates.prehookFunction
prehook(::Plugin, ::Template, pkg_dir::AbstractString)

Stage 1 of the package generation process (the "before" stage, in general). At this point, pkg_dir is an empty directory that will eventually contain the package, and neither the hooks nor the posthooks have run.

Note

pkg_dir only stays empty until the first plugin chooses to create a file. See also: priority.

source
PkgTemplates.hookFunction
hook(::Plugin, ::Template, pkg_dir::AbstractString)

Stage 2 of the package generation pipeline (the "main" stage, in general). At this point, the prehooks have run, but not the posthooks.

pkg_dir is the directory in which the package is being generated (so basename(pkg_dir) is the package name).

Note

You usually shouldn't implement this function for FilePlugins. If you do, it should probably invoke the generic method (otherwise, there's not much reason to subtype FilePlugin).

source
PkgTemplates.posthookFunction
posthook(::Plugin, ::Template, pkg_dir::AbstractString)

Stage 3 of the package generation pipeline (the "after" stage, in general). At this point, both the prehooks and hooks have run.

source
PkgTemplates.priorityFunction
priority(::Plugin, ::Union{typeof(prehook), typeof(hook), typeof(posthook)}) -> Int

Determines the order in which plugins are processed (higher goes first). The default priority (DEFAULT_PRIORITY), is 1000.

You can implement this function per-stage (by using ::typeof(hook), for example), or for all stages by simply using ::Function.

source

Plugin Walkthrough

Concrete types that subtype Plugin directly are free to do almost anything. To understand how they're implemented, let's look at simplified versions of two plugins: Documenter to explore templating, and Git to further clarify the multi-stage pipeline.

Example: Documenter

@plugin struct Documenter <: Plugin
    make_jl::String = default_file("docs", "make.jl")
    index_md::String = default_file("docs", "src", "index.md")
end

gitignore(::Documenter) = ["/docs/build/"]

badges(::Documenter) = [
    Badge(
        "Stable",
        "https://img.shields.io/badge/docs-stable-blue.svg",
        "https://{{{USER}}}.github.io/{{{PKG}}}.jl/stable",
    ),
    Badge(
        "Dev",
        "https://img.shields.io/badge/docs-dev-blue.svg",
        "https://{{{USER}}}.github.io/{{{PKG}}}.jl/dev",
    ),
]

view(p::Documenter, t::Template, pkg::AbstractString) = Dict(
    "AUTHORS" => join(t.authors, ", "),
    "PKG" => pkg,
    "REPO" => "$(t.host)/$(t.user)/$pkg.jl",
    "USER" => t.user,
)

function hook(p::Documenter, t::Template, pkg_dir::AbstractString)
    pkg = basename(pkg_dir)
    docs_dir = joinpath(pkg_dir, "docs")

    make = render_file(p.make_jl, combined_view(p, t, pkg), tags(p))
    gen_file(joinpath(docs_dir, "make.jl"), make)
    
    index = render_file(p.index_md, combined_view(p, t, pkg), tags(p))
    gen_file(joinpath(docs_dir, "src", "index.md"), index)

    # What this function does is not relevant here.
    create_documentation_project()
end

The @plugin macro defines some helpful methods for us. Inside of our struct definition, we're using default_file to refer to files in this repository.

PkgTemplates.@pluginMacro
@plugin struct ... end

Define a plugin subtype with keyword constructors and default values.

For details on the general syntax, see Parameters.jl.

There are a few extra restrictions:

  • Before using this macro, you must have imported @with_kw_noshow via using PkgTemplates: @with_kw_noshow
  • The type must be a subtype of Plugin (or one of its abstract subtypes)
  • The type cannot be parametric
  • All fields must have default values

Example

using PkgTemplates: @plugin, @with_kw_noshow, Plugin
@plugin struct MyPlugin <: Plugin
    x::String = "hello!"
    y::Union{Int, Nothing} = nothing
end

Implementing @plugin Manually

If for whatever reason, you are unable to meet the criteria outlined above, you can manually implement the methods that @plugin would have created for you. This is only mandatory if you want to use your plugin in interactive mode.

Keyword Constructors

If possible, use @with_kw_noshow to create a keyword constructor for your type. Your type must be capable of being instantiated with no arguments.

Default Values

If your type's fields have sensible default values, implement defaultkw like so:

using PkgTemplates: PkgTemplates, Plugin
struct MyPlugin <: Plugin
    x::String
end
PkgTemplates.defaultkw(::Type{MyPlugin}, ::Val{:x}) = "my default"

Remember to add a method to the function belonging to PkgTemplates, rather than creating your own function that PkgTemplates won't see.

If your plugin's fields have no sane defaults, then you'll need to implement prompt appropriately instead.

source
PkgTemplates.default_fileFunction
default_file(paths::AbstractString...) -> String

Return a path relative to the default template file directory (PkgTemplates/templates).

source

The first method we implement for Documenter is gitignore, so that packages created with this plugin ignore documentation build artifacts.

PkgTemplates.gitignoreFunction
gitignore(::Plugin) -> Vector{String}

Return patterns that should be added to .gitignore. These are used by the Git plugin.

By default, an empty list is returned.

source

Second, we implement badges to add a couple of badges to new packages' README files.

PkgTemplates.badgesFunction
badges(::Plugin) -> Union{Badge, Vector{Badge}}

Return a list of Badges, or just one, to be added to README.md. These are used by the Readme plugin to add badges to the README.

By default, an empty list is returned.

source
PkgTemplates.BadgeType
Badge(hover::AbstractString, image::AbstractString, link::AbstractString)

Container for Markdown badge data. Each argument can contain placeholders, which will be filled in with values from combined_view.

Arguments

  • hover::AbstractString: Text to appear when the mouse is hovered over the badge.
  • image::AbstractString: URL to the image to display.
  • link::AbstractString: URL to go to upon clicking the badge.
source

These two functions, gitignore and badges, are currently the only "special" functions for cross-plugin interactions. In other cases, you can still access the Template's plugins to depend on the presence/properties of other plugins via getplugin, although that's less powerful.

PkgTemplates.getpluginFunction
getplugin(t::Template, ::Type{T<:Plugin}) -> Union{T, Nothing}

Get the plugin of type T from the template t, if it's present.

source

Third, we implement view, which is used to fill placeholders in badges and rendered files.

PkgTemplates.viewFunction
view(::Plugin, ::Template, pkg::AbstractString) -> Dict{String, Any}

Return the view to be passed to the text templating engine for this plugin. pkg is the name of the package being generated.

For FilePlugins, this is used for both the plugin badges (see badges) and the template file (see source). For other Plugins, it is used only for badges, but you can always call it yourself as part of your hook implementation.

By default, an empty Dict is returned.

source

Finally, we implement hook, which is the real workhorse for the plugin. Inside of this function, we generate a couple of files with the help of a few more text templating functions.

PkgTemplates.render_fileFunction
render_file(file::AbstractString view::Dict{<:AbstractString}, tags=nothing) -> String

Render a template file with the data in view. tags should be a tuple of two strings, which are the opening and closing delimiters, or nothing to use the default delimiters.

source
PkgTemplates.render_textFunction
render_text(text::AbstractString, view::Dict{<:AbstractString}, tags=nothing) -> String

Render some text with the data in view. tags should be a tuple of two strings, which are the opening and closing delimiters, or nothing to use the default delimiters.

source
PkgTemplates.gen_fileFunction
gen_file(file::AbstractString, text::AbstractString)

Create a new file containing some given text. Trailing whitespace is removed, and the file will end with a newline.

source
PkgTemplates.combined_viewFunction
combined_view(::Plugin, ::Template, pkg::AbstractString) -> Dict{String, Any}

This function combines view and user_view for use in text templating. If you're doing manual file creation or text templating (i.e. writing Plugins that are not FilePlugins), then you should use this function rather than either of the former two.

Note

Do not implement this function yourself! If you're implementing a plugin, you should implement view. If you're customizing a plugin as a user, you should implement user_view.

source
PkgTemplates.tagsFunction
tags(::Plugin) -> Tuple{String, String}

Return the delimiters used for text templating. See the Citation plugin for a rare case where changing the tags is necessary.

By default, the tags are "{{" and "}}".

source

For more information on text templating, see the FilePlugin Walkthrough and the section on Custom Template Files.

Example: Git

struct Git <: Plugin end

priority(::Git, ::typeof(posthook)) = 5

function validate(::Git, ::Template)
    foreach(("user.name", "user.email")) do k
        if isempty(LibGit2.getconfig(k, ""))
            throw(ArgumentError("Git: Global Git config is missing required value '$k'"))
        end
    end
end

function prehook(::Git, t::Template, pkg_dir::AbstractString)
    LibGit2.with(LibGit2.init(pkg_dir)) do repo
        LibGit2.commit(repo, "Initial commit")
        pkg = basename(pkg_dir)
        url = "https://$(t.host)/$(t.user)/$pkg.jl"
        close(GitRemote(repo, "origin", url))
    end
end

function hook(::Git, t::Template, pkg_dir::AbstractString)
    ignore = mapreduce(gitignore, append!, t.plugins)
    unique!(sort!(ignore))
    gen_file(joinpath(pkg_dir, ".gitignore"), join(ignore, "\n"))
end

function posthook(::Git, ::Template, pkg_dir::AbstractString)
    LibGit2.with(GitRepo(pkg_dir)) do repo
        LibGit2.add!(repo, ".")
        LibGit2.commit(repo, "Files generated by PkgTemplates")
    end
end

We didn't use @plugin for this one, because there are no fields. Validation and all three hooks are implemented:

  • validate makes sure that all required Git configuration is present.
  • prehook creates the Git repository for the package.
  • hook generates the .gitignore file, using the special gitignore function.
  • posthook adds and commits all the generated files.

As previously mentioned, we use priority to make sure that we wait until all other plugins are finished their work before committing files.

Hopefully, this demonstrates the level of control you have over the package generation process when developing plugins, and when it makes sense to exercise that power!

FilePlugin Walkthrough

Most of the time, you don't really need all of the control that we showed off above. Plugins that subtype FilePlugin perform a much more limited task. In general, they just generate one templated file.

To illustrate, let's look at the Citation plugin, which creates a CITATION.bib file.

@plugin struct Citation <: FilePlugin
    file::String = default_file("CITATION.bib")
end

source(p::Citation) = p.file
destination(::Citation) = "CITATION.bib"

tags(::Citation) = "<<", ">>"

view(::Citation, t::Template, pkg::AbstractString) = Dict(
    "AUTHORS" => join(t.authors, ", "),
    "MONTH" => month(today()),
    "PKG" => pkg,
    "URL" => "https://$(t.host)/$(t.user)/$pkg.jl",
    "YEAR" => year(today()),
)

Similar to the Documenter example above, we're defining a keyword constructor, and assigning a default template file from this repository. This plugin adds nothing to .gitignore, and it doesn't add any badges, so implementations for gitignore and badges are omitted.

First, we implement source and destination to define where the template file comes from, and where it goes. These functions are specific to FilePlugins, and have no effect on regular Plugins by default.

PkgTemplates.sourceFunction
source(::FilePlugin) -> Union{String, Nothing}

Return the path to a plugin's template file, or nothing to indicate no file.

By default, nothing is returned.

source
PkgTemplates.destinationFunction
destination(::FilePlugin) -> String

Return the destination, relative to the package root, of a plugin's configuration file.

This function must be implemented.

source

Next, we implement tags. We briefly saw this function earlier, but in this case it's necessary to change its behaviour from the default. To see why, it might help to see the template file in its entirety:

@misc{<<&PKG>>.jl,
	author  = {<<&AUTHORS>>},
	title   = {<<&PKG>>.jl},
	url     = {<<&URL>>},
	version = {v0.1.0},
	year    = {<<&YEAR>>},
	month   = {<<&MONTH>>}
}

Because the file contains its own {} delimiters, we need to use different ones for templating to work properly.

Finally, we implement view to fill in the placeholders that we saw in the template file.

Doing Extra Work With FilePlugins

Notice that we didn't have to implement hook for our plugin. It's implemented for all FilePlugins, like so:

function render_plugin(p::FilePlugin, t::Template, pkg::AbstractString)
    return render_file(source(p), combined_view(p, t, pkg), tags(p))
end

function hook(p::FilePlugin, t::Template, pkg_dir::AbstractString)
    source(p) === nothing && return
    pkg = basename(pkg_dir)
    path = joinpath(pkg_dir, destination(p))
    text = render_plugin(p, t, pkg)
    gen_file(path, text)
end

But what if we want to do a little more than just generate one file?

A good example of this is the Tests plugin. It creates runtests.jl, but it also modifies the Project.toml to include the Test dependency.

Of course, we could use a normal Plugin, but it turns out there's a way to avoid that while still getting the extra capbilities that we want.

The plugin implements its own hook, but uses invoke to avoid duplicating the file creation code:

@plugin struct Tests <: FilePlugin
    file::String = default_file("runtests.jl")
end

source(p::Tests) = p.file
destination(::Tests) = joinpath("test", "runtests.jl")
view(::Tests, ::Template, pkg::AbstractString) = Dict("PKG" => pkg)

function hook(p::Tests, t::Template, pkg_dir::AbstractString)
    # Do the normal FilePlugin behaviour to create the test script.
    invoke(hook, Tuple{FilePlugin, Template, AbstractString}, p, t, pkg_dir)
    # Do some other work.
    add_test_dependency()
end

There is also a default validate implementation for FilePlugins, which checks that the plugin's source file exists, and throws an ArgumentError otherwise. If you want to extend the validation but keep the file existence check, use the invoke method as described above.

For more examples, see the plugins in the Continuous Integration (CI) and Code Coverage sections.

Supporting Interactive Mode

When it comes to supporting interactive mode for your custom plugins, you have two options: write your own interactive method, or use the default one. If you choose the first option, then you are free to implement the method however you want. If you want to use the default implementation, then there are a few functions that you should be aware of, although in many cases you will not need to add any new methods.

PkgTemplates.interactiveFunction
interactive(T::Type{<:Plugin}) -> T

Interactively create a plugin of type T. Implement this method and ignore other related functions only if you want completely custom behaviour.

source
PkgTemplates.promptFunction
prompt(::Type{P}, ::Type{T}, ::Val{name::Symbol}) -> Any

Prompts for an input of type T for field name of plugin type P. Implement this method to customize particular fields of particular types.

source
PkgTemplates.customizableFunction
customizable(::Type{<:Plugin}) -> Vector{Pair{Symbol, DataType}}

Return a list of keyword arguments that the given plugin type accepts, which are not fields of the type, and should be customizable in interactive mode. For example, for a constructor Foo(; x::Bool), provide [x => Bool]. If T has fields which should not be customizable, use NotCustomizable as the type.

source
PkgTemplates.input_tipsFunction
input_tips(::Type{T}) -> Vector{String}

Provide some extra tips to users on how to structure their input for the type T, for example if multiple delimited values are expected.

source
PkgTemplates.convert_inputFunction
convert_input(::Type{P}, ::Type{T}, s::AbstractString) -> T

Convert the user input s into an instance of T for plugin of type P. A default implementation of T(s) exists.

source

Miscellaneous Tips

Writing Template Files

For an overview of writing template files for Mustache.jl, see Custom Template Files in the user guide.

Predicates

There are a few predicate functions for plugins that are occasionally used to answer questions like "does this Template have any code coverage plugins?". If you're implementing a plugin that fits into one of the following categories, it would be wise to implement the corresponding predicate function to return true for instances of your type.

PkgTemplates.needs_usernameFunction
needs_username(::Plugin) -> Bool

Determine whether or not a plugin needs a Git hosting service username to function correctly. If you are implementing a plugin that uses the user field of a Template, you should implement this function and return true.

source
PkgTemplates.is_ciFunction
is_ci(::Plugin) -> Bool

Determine whether or not a plugin is a CI plugin. If you are adding a CI plugin, you should implement this function and return true.

source
PkgTemplates.is_coverageFunction
is_coverage(::Plugin) -> Bool

Determine whether or not a plugin is a coverage plugin. If you are adding a coverage plugin, you should implement this function and return true.

source

Formatting Version Numbers

When writing configuration files for CI services, working with version numbers is often needed. There are a few convenience functions that can be used to make this a little bit easier.

PkgTemplates.format_versionFunction
format_version(v::Union{VersionNumber, AbstractString}) -> String

Strip everything but the major and minor release from a VersionNumber. Strings are left in their original form.

source
PkgTemplates.collect_versionsFunction
collect_versions(t::Template, versions::Vector) -> Vector{String}

Combine t's Julia version with versions, and format them as major.minor. This is useful for creating lists of versions to be included in CI configurations.

source

Testing

If you write a cool new plugin that could be useful to other people, or find and fix a bug, you're encouraged to open a pull request with your changes. Here are some testing tips to ensure that your PR goes through as smoothly as possible.

Updating Reference Tests & Fixtures

If you've added or modified plugins, you should update the reference tests and the associated test fixtures. In test/reference.jl, you'll find a "Reference tests" test set that basically generates a bunch of packages, and then checks each file against a reference file, which is stored somewhere in test/fixtures. Note the reference tests only run on one specific version of Julia; check test/runtests.jl to see the current version used.

For new plugins, you should add an instance of your plugin to the "All plugins" and "Wacky options" test sets, then run the tests with Pkg.test. They should pass, and there will be new files in test/fixtures. Check them to make sure that they contain exactly what you would expect!

For changes to existing plugins, update the plugin options appropriately in the "Wacky options" test set. Failing tests will give you the option to review and accept changes to the fixtures, updating the files automatically for you.

Updating "Show" Tests

Depending on what you've changed, the tests in test/show.jl might fail. To fix those, you'll need to update the expected value to match what is actually displayed in a Julia REPL (assuming that the new value is correct).