NanoLoc

Welcome to the documentation of NanoLoc! NanoLoc is a julia package to explore, benchmark, and develop novel nanoscopy methods that work by probing fluorescent markers via structured illumination (SI).

Note

This package is currently under heavy development. Breaking changes are to be expected.

Background

In recent years, fluorescence nanoscopy, or super-resolution microscopy, has went through several conceptual and experimental breakthroughs. Some of the developed modalities, like MINFLUX or MINSTED, localize single fluorescent markers by applying a series of purposefully selected illumination patterns to the sample. The photon responses detected upon these structured illuminations (i.e., how many photons are observed in which timeframe?) tell us about the position and excitability of the marker.

At the expense of only a handfull of photons, localization accuracies of 10 nm and below become possible. This photon efficiency far surpasses the performance of traditional microscopy and earlier forms of nanoscopy.

While the established techniques work remarkably well in practice, the fundamental efficiency and accuracy limits of this class of SI-nanoscopy methods are currently not well understood. In particular, it is unclear what can be gained by measuring in an optimal way, given realistic physical constraints on the illumination patterns.

Besides the potential practical consequences, there is another reason why SI-nanoscopy serves as an interesting testbed and playground for novel localization algorithms: The statistical model governing the photon emission is, in good approximation, surprisingly simple, such that explicit likelihoods for different measurement designs can easily be implemented and efficiently be computed. Thus, both explicit statistical modeling approaches (in particular techniques for sequential Bayesian updates and design optimizations) as well as various modern machine learning techniques can jointly be tested on this problem.

The goal of NanoLoc is to provide a convenient, flexible, and easily extensible framework to facilitate research along these lines in a modern and dynamic programming language.

Installation

Currently, NanoLoc is not yet registered as an official julia package. It can be installed by running

using Pkg
Pkg.add(url = "https://gitlab.gwdg.de/staudt1/nanoloc.jl")

in a julia REPL. If you not only want to use, but want to help work on the package, use

using Pkg
Pkg.develop(url = "https://gitlab.gwdg.de/staudt1/nanoloc.jl")

instead. This way, the git repository is cloned to the local folder ~/.julia/dev/NanoLoc on your system. Changes in this folder will be reflected when you use NanoLoc in your julia code (via using NanoLoc or import NanoLoc).

NanoLoc is currently developed against julia version 1.10. While older version should work as well in many cases, specific features, like GPU support via package extensions, require at least julia 1.9.

Quickstart

In order to run a standard NanoLoc localization simulation, three ingredients are needed:

  • the parameter of the statistical model. For the localization of a single fluorescent molecule, this is the position x, the brightness (or excitability) alpha, and the noise level noise. Such a parameter object can be created via the constructor Param.
  • a policy for picking measurement designs. These designs may depend on information that has been accumulated before, like previous measurement results. A nice example for an adaptive policy is Minsted, which mimics the MINSTED nanoscopy method.
  • an inference method that specifies how to integrate novel observations into our current beliefs about the system (i.e., the marker position and brightness). For example, the inference method BayesGrid assumes a discrete grid-based approximation of the parameter space and uses Bayesian posterior updates on this grid when new measurements are to be integrated.

In NanoLoc, we can realize these choices as follows:

using NanoLoc

param = Param((25, 75), alpha = 1.0, noise = 0.1)
policy = Minsted((50,50), fwhm_max = 100, fwhm_min = 50)
method = BayesGrid(0:1:100, 0:1:100, alpha = :known, noise = :known)

We first create a model parameter at the position x = (50, 50), with brightness alpha = 1.0 and noise level noise = 0.1.

Afterwards, we create a policy object: a Minsted policy initially centered at (50, 50), with an FWHM that will start at 100 and should drop to 50 during the course of the experiment.

Finally, as an inference method, we choose a BayesGrid that covers a 2-dimensional spatial grid with support points {0, 1, ..., 100} x {0, 1, ..., 100}. As is indicated by the keyword arguments, the method we created assumes the values of alpha and noise to be known, i.e., they will be derive from param during the simulation.

We can now run a simulation:

trace = localize(policy, method, param, stop = t -> t.photons >= 50)
SimulatedTrace 
 policy   minsted
 method   BayesGrid(101×101×1×1)
 upgrades 1
 steps    50
 photons  50 / 6 bg
 param    P(25.0, 75.0, a:1.0, n:0.1)
 estimate P(27.4, 71.2, a:1.0, n:0.1)
 err/std  4.52 / 6.53

In this line, we encounter the main entry point for NanoLoc simulations: the method localize, which creates an object of type SimulatedTrace. Expressed in pseudo-code, localize roughly works as follows:

function localize(policy, method, param; stop)
  # initialize the trace object
  trace = init_trace(method)

  # run the following code until the stop-criterion is satisfied
  while !stop(trace)
    # let the policy decide where and how to measure next
    design = policy(trace)

    # make a measurement / draw an observation from the statistical model
    obs = rand(design, param)

    # use the method to incorporate the novel information into the current inference state
    trace.state = update(method, trace.state, design, observation)
  end
end

Besides the current inference state, the trace object also stores other information gathered during the localization simulation. For example, it tracks the total number of photons accumulated (accessible via trace.photons). The argument stop in the example above uses this information to halt the simulation as soon as the internally generated trace (the anonymous function argument t) has recorded 50 (or more) photons.

In fact, a number of metrics are recorded at every single step of the trace generation. These metrics can be accessed via trace[step], where step is an integer between 0 (before the first measurement) and trace.steps (after the final measurement).

For example, the following code retrieves the position estimate, its (Euclidean) localization error, and the signal-to-background ratio at step 20:

trace[20].est
Param{2, Float32}
 x     (40.21, 78.57)
 alpha 1.0
 noise 0.1
trace[20].errx
15.627344f0
trace[20].sbr
8.923148f0

Thus, while we have achieved a localization accuracy trace[end].errx of about 4.5 after 50 photons, it was only roughly 15.6 after 20 photons. Out of the 50 photons in total, trace[end].bgphotons == 6 were noise photons. The remaining 44 ones (trace[end].signalphotons) were signal photons.

Next steps

The example above should give you an impression of how NanoLoc works and what it can potentially help you with. However, it only scratches the surface of the available functionality.

  • More advanced usage examples are provided at Examples.
  • You can learn more about how to handle Traces and Stacks, the latter of which are collections of traces that share a policy and method.
  • To see how measurement designs work, how they power policies, and how you can customize and extend both of them, consult Designs and Policies.
  • Finally, while Bayesian grids are good and reliable choices for various inference tasks, they are also a brute-force approach and not terribly efficient. To learn about alternative methods how knowledge about the marker can be represented and tracked, head to Inference.