Packaging an External Library#

The Defining a Library guide shows how to subclass Design, StdCellLibrary, and PDK to describe a piece of IP. That is enough when the library class lives alongside the design that uses it. Once you want to share a library, across projects, across a team, or as a versioned deliverable, you package it as a standalone, pip-installable Python distribution.

This guide covers that packaging layer: the project layout, the pyproject.toml, and how bundled data files are located no matter where the package is installed. A single package can bundle libraries, PDKs, flows, and even custom tool drivers, and expose them through a target. For the details of the individual module classes, this guide links back to libraries, pdks, flows, tools, and targets. Throughout, the package is given the generic name mylib so you can substitute your own.

Why package a library#

Distributing a library as its own package buys you:

  • Reuse — any project can pip install mylib and import it, instead of copying files into each design.

  • Versioning — the package carries a version, so a design can pin the exact library revision it was signed off against.

  • Data locality — data files (LEF, GDS, .lib, …) travel with the package and resolve automatically, whether the package is installed from PyPI or in editable mode from a local checkout.

Package layout#

A library package is an ordinary Python distribution with an importable module at its root. A typical layout mirrors the categories of module you provide:

mylib/                    # repository root
├── pyproject.toml        # build + packaging configuration
├── MANIFEST.in           # controls which files land in the sdist
├── README.md
├── LICENSE
└── mylib/                # the importable Python package
    ├── __init__.py       # package init (optionally a shared dataroot mixin)
    ├── _version.py       # auto-generated by setuptools_scm
    ├── libs/             # standard-cell / hard IP (StdCellLibrary subclasses)
    ├── pdks/             # PDK definitions (siliconcompiler.PDK subclasses)
    ├── flows/            # custom Flowgraph definitions
    ├── tools/            # custom tool/task drivers
    └── targets/          # targets bundling any of the above

Every module subpackage is optional — include only what you ship. A package might provide just a library, just a PDK, just a flow or tool driver, or any combination, and expose it through a target.

Small data files (a hand-edited LEF, a black-box Verilog stub) can be committed directly under the relevant subpackage, e.g. mylib/libs/mymacro/mymacro.lef. Large or proprietary data (full foundry decks) is usually kept outside the package and referenced through an environment variable, see Two dataroot strategies.

The pyproject.toml#

The build configuration does two jobs: build the package and derive its version.

[build-system]
requires = ["setuptools >= 80", "setuptools_scm[toml] >= 8"]
build-backend = "setuptools.build_meta"

# Derive the version from git tags and write it into the package.
[tool.setuptools_scm]
version_file = "mylib/_version.py"

[project]
name = "mylib"
description = "Example external library for SiliconCompiler."
readme = "README.md"
requires-python = ">= 3.10"
# For proprietary IP, use a license reference instead of an SPDX id, e.g.
# license = "LicenseRef-MyCompany-Proprietary"
license = "Apache-2.0"
license-files = ["LICENSE"]
dynamic = ["version"]
dependencies = [
    "siliconcompiler == 0.38.0",
    # Depend on other libraries you build on, e.g.:
    # "lambdapdk",
]

[tool.setuptools]
include-package-data = true

Points worth calling out:

  • Versioning — setuptools_scm derives the package version from git tags and writes it into mylib/_version.py on every build, so each release is stamped with a version that designs can pin against. Because that file is generated, add it to .gitignore rather than committing it:

    # .gitignore
mylib/_version.py

  • ``include-package-data = true`` — together with MANIFEST.in, this ensures committed data files ship inside the wheel. Prune what you do not need (tests, examples) and explicitly include your package sources and data:

    # MANIFEST.in
prune tests
prune examples
recursive-include mylib *.py
recursive-include mylib *.lef *.gds *.v

  • Pinning SiliconCompiler — pin siliconcompiler in dependencies to the exact release you develop against using an == constraint; the current release is 0.38.0.

Note

More advanced packages can also register SiliconCompiler entry points to plug into documentation source links, show/open tasks, and tool-install helpers (the siliconcompiler.docs, siliconcompiler.showtask, and siliconcompiler.install groups), plus [project.scripts] for CLI apps.

Locating data files#

For SiliconCompiler to find a module’s data files no matter where the package is installed, each module registers a dataroot. The simplest approach anchors the dataroot to the module’s own source file: passing __file__ to set_dataroot roots the data at the directory containing that module, so files are added relative to it.

self.set_dataroot(f"mylib-{self.name}", __file__)

Giving the dataroot a name unique to the module (here mylib-<name>) keeps it from clashing when several libraries are combined into one project.

Note

If you would rather share a single dataroot across the whole package — for example to reference files that live in sibling subpackages — define a small mixin that every module inherits and point it at the installed package:

# mylib/__init__.py
from siliconcompiler.schema_support.pathschema import PathSchema


class _MyLibPath(PathSchema):
    def __init__(self):
        super().__init__()
        self.set_dataroot("mylib", "python://mylib")

The python://mylib dataroot resolves to the installed location of the mylib package. To additionally fetch large data files that are not shipped in the wheel, give the dataroot a git fallback keyed to the package version with PythonPathResolver.set_dataroot.

Two dataroot strategies#

Real libraries commonly mix two kinds of dataroot:

  1. Packaged data — the per-module dataroot rooted at __file__ above, for files committed to and shipped with the package. Use this for small files (edited LEFs, black-box stubs, RTL stubs).

  2. External data — foundry decks are often too large or too restricted to commit. Point a second dataroot at an environment variable so users supply the data out-of-band:

    self.set_dataroot("foundry", "$FOUNDRY_ROOT/tech/mylib/v1p0")

    Files added under this dataroot resolve against $FOUNDRY_ROOT at run time. Whenever a module relies on an environment variable like this, set a default for it in the target so users are not left to configure it by hand.

Defining modules inside the package#

Libraries, PDKs, flows, and tool drivers are defined exactly as in the libraries, pdks, flows, and tools guides. The only packaging addition is that any module shipping data files sets a dataroot from __file__ (and typically mixes in CachedSchema) so those files travel with the package. A hard-IP library looks like:

# mylib/libs/mymacro/__init__.py
from siliconcompiler import StdCellLibrary
from siliconcompiler.schema import CachedSchema
from siliconcompiler.tools.openroad import OpenROADStdCellLibrary
from siliconcompiler.tools.klayout import KLayoutLibrary

from mylib.pdks import mypdk


class MyMacro(OpenROADStdCellLibrary, KLayoutLibrary,
              StdCellLibrary, CachedSchema):
    def __init__(self):
        super().__init__()
        self.set_name("mymacro")
        self.package.set_vendor("MyVendor")
        self.package.set_version("v1p0")

        # Root data files at this module's own directory.
        self.set_dataroot(f"mylib-{self.name}", __file__)

        # Associate the PDK this IP targets.
        self.add_asic_pdk(mypdk.MyPDK())

        # Files committed next to this module, added relative to its directory.
        with self.active_dataroot(f"mylib-{self.name}"):
            with self.active_fileset("models.physical"):
                self.add_file("mymacro.lef")
                self.add_file("mymacro.gds")
                self.add_asic_aprfileset()

        # Timing models come from an external dataroot. Keep each format in
        # its own fileset, tagged by corner and delay model with
        # add_asic_libcornerfileset(corner, model). Liberty (.lib) uses a
        # model like "nldm"; a compiled binary timing model (here ".bin")
        # uses the "<model>-bin" variant.
        self.set_dataroot("foundry", "$FOUNDRY_ROOT/tech/mylib/v1p0")
        with self.active_dataroot("foundry"):
            for corner in ("slow", "typical", "fast"):
                with self.active_fileset(f"models.timing.{corner}.nldm"):
                    self.add_file(f"lib/mymacro_{corner}.lib.gz")
                    self.add_asic_libcornerfileset(corner, "nldm")

                # Compiled binary timing model for the same corner.
                with self.active_fileset(f"models.timing.{corner}.nldm-bin"):
                    self.add_file(f"bin/mymacro_{corner}.bin", filetype="bin")
                    self.add_asic_libcornerfileset(corner, "nldm-bin")

The tool mixins (OpenROADStdCellLibrary, KLayoutLibrary, …) attach tool-specific parameters so the library carries everything a flow needs; see Tool Extension Classes for the full list. PDKs follow the same recipe with PDK and the tool *PDK mixins; see pdks.

A tool driver is a Task subclass. It names its tool and task, declares the executable, and ships any reference scripts alongside the module:

# mylib/tools/mytool/__init__.py
from siliconcompiler import Task


class MyToolTask(Task):
    def tool(self):
        return "mytool"

    def task(self):
        return "convert"

    def parse_version(self, stdout):
        return stdout.strip()

    def setup(self):
        super().setup()
        self.set_exe("mytool", vswitch="--version", format="tcl")
        self.add_version(">=1.0.0")

        # Reference scripts committed next to this module.
        self.set_dataroot("mytool-scripts", __file__)
        with self.active_dataroot("mytool-scripts"):
            self.set_refdir("scripts")
        self.set_script("convert.tcl")

    def runtime_options(self):
        options = super().runtime_options()
        # Each token is its own list item — never "-flag value" as one string.
        options.append("-batch")
        options.extend(["-log", "convert.log"])
        return options

A task with no external executable overrides run() (returning an exit code) instead of declaring an exe; everything else about the class is the same. See tools for the full task API.

A flow is a Flowgraph subclass that wires tasks into a graph of steps with node() and edge() — and it can reference both built-in tasks and the package’s own:

# mylib/flows/myflow.py
from siliconcompiler import Flowgraph
from siliconcompiler.tools.slang.elaborate import Elaborate

from mylib.tools.mytool import MyToolTask


class MyFlow(Flowgraph):
    def __init__(self, name: str = "myflow"):
        super().__init__(name)
        self.node("elaborate", Elaborate())
        self.node("convert", MyToolTask())
        self.edge("elaborate", "convert")

See flows for composing larger graphs.

ASIC timing models and the delay model#

Standard-cell timing ships in more than one format. Liberty (.lib) is read by the open-source tools; some libraries also provide a compiled, binary timing model (here .bin) that a tool can load without parsing Liberty. A library keeps each format in its own fileset, tagged with a corner and a delay model using add_asic_libcornerfileset(corner, model) (see the library example above). By convention the compiled variant of a delay model is named <model>-bin:

Delay model

Fileset

Files

nldm

models.timing.<corner>.nldm

Liberty (.lib)

nldm-bin

models.timing.<corner>.nldm-bin

compiled (.bin)

ccs

models.timing.<corner>.ccs

CCS Liberty (.lib)

ccs-bin

models.timing.<corner>.ccs-bin

compiled (.bin)

A target chooses one with project.set_asic_delaymodel("nldm") (or "nldm-bin" for a flow that reads the compiled files). delaymodel is a free-form string, so <model>-bin works purely by matching the fileset name you registered. Tool drivers resolve the files by looking up (corner, delaymodel); for how a driver consumes these filesets in Python and TCL, see Reading ASIC standard-cell timing.

Bundling into a target#

To give users a one-call setup, provide a target function that assembles the PDK, libraries, and flow — and sets a default for any environment variable the modules depend on:

# mylib/targets/mytarget.py
import os

from siliconcompiler import ASIC

from mylib.pdks import mypdk
from mylib.libs import mymacro
from mylib.flows.myflow import MyFlow


def mytarget(project: ASIC):
    # Default the external dataroot only if the user has not already set it,
    # in the environment or on the project.
    if os.getenv("FOUNDRY_ROOT") is None and \
            project.option.get_env("FOUNDRY_ROOT") is None:
        project.option.set_env("FOUNDRY_ROOT", "/opt/foundry")

    project.set_pdk(mypdk.MyPDK())
    project.set_mainlib(mymacro.MyMacro())
    project.set_flow(MyFlow())

    # Pick the timing view the tools read (see the delay-model section above);
    # use "nldm-bin" instead for a flow that reads the compiled timing model.
    project.set_asic_delaymodel("nldm")

If a module points a dataroot at an environment variable such as $FOUNDRY_ROOT, setting it here with project.option.set_env means users get a working default without configuring anything themselves.

Publishing to PyPI#

For an open-source library, publishing to PyPI lets anyone install it with a plain pip install mylib. Build the wheel and upload it:

python3 -m build                 # build the wheel and sdist into dist/
python3 -m twine upload dist/*   # publish to PyPI

Most projects prefer to automate this with a CI workflow — for example a GitHub Action using pypa/gh-action-pypi-publish — that builds and publishes on every tagged release. Proprietary libraries skip PyPI and are shared instead through a private package index or straight from the git remote (see the VCS install form below).

Installing and using the library#

Once published, the library is installed like any Python package:

pip install mylib
# for local development against your working tree:
pip install -e .
# or install a specific tagged revision straight from a git remote:
pip install "mylib @ git+https://company.com/git/[email protected]"

The last form uses pip’s VCS direct-reference syntax, which is handy for private libraries hosted on an internal git server.

The package is consumed either through its target or by adding its modules directly:

import siliconcompiler
from mylib.libs import mymacro
from mylib.targets.mytarget import mytarget

project = siliconcompiler.ASIC()

# Option A: one-call setup via the target.
mytarget(project)

# Option B: attach modules directly.
project.add_asiclib(mymacro.MyMacro())

# project.run()

Because each library roots its dataroot at its own __file__, all of its data files resolve automatically, whether it was installed from PyPI or checked out in editable mode.

Reference#