Skip to content

Control layers and references#

Three layer types steer a generation without changing the prompt: control layers (structure), regional guidance (where text applies) and reference images (look). They combine freely with raster layers and inpainting.

Control layers#

A control layer conditions the generation on a map derived from an image: edges, depth, pose, a colour map or a tile. Add one from the layer menu or by dropping an image As Control Layer, then set:

  • Control model: a ControlNet or Control LoRA installed for the base model you generate with. Union models cover several map types; the Z-Image ControlNets Union and Tile ship in the starter catalogue.
  • Processor (filter): how the map is made from the image. Canny for line edges (low and high thresholds), Depth Anything (model size small to large), colour map (tile size), content shuffle, and the rest of the filter list. Auto Process re-runs the filter when you change a setting; Process runs it once. Or paint the map yourself.
  • Weight: how strongly the map is enforced. 0.75 to 1 is typical; above 1 forces the structure at the cost of detail.
  • Begin and end: the fraction of denoising during which the control applies. Ending at 0.6 to 0.8 leaves the model room to finish details.
  • Control mode: balanced, prompt priority or control priority.
  • Resize mode: how a map of a different size is fitted to the bounding box.

A tile control layer with the tile model is what the Upscaling tab uses under the hood; on the canvas it lets you regenerate detail while keeping the layout.

Regional guidance#

A regional guidance layer pairs a painted mask with its own prompt, so one part of the image follows different text. Paint the region, add a positive prompt (and a negative one if the model uses it) or a regional reference image. Auto Negative adds the region's prompt to the negative prompt of the rest of the image, which keeps the concept from leaking outside the mask. Overlapping regions blend.

Regional guidance is the tool for "a red car on the left, a blue car on the right": describe the shared scene in the global prompt and the specifics per region.

Reference images#

A global reference image conditions the whole generation on a picture through an IP-Adapter. A regional reference image does the same inside a regional guidance mask.

  • Model: the IP-Adapter for your base model. The catalogue offers Standard (looser), Precise (Plus) and Face variants for SD 1.5 and SDXL, and the XLabs adapter for FLUX.
  • Method: Full transfers subject and style, Style only the look, Composition only the layout.
  • Weight and begin/end work like control layers. Start at 0.6 to 0.8.
  • Pull Bbox into Reference Image uses what is currently under the bounding box as the reference, handy for iterating on a style you just generated.

For a face that must hold across many images, a reference image is a good start but a trained LoRA is the reliable tool: see Character LoRAs for video and Train an image LoRA for images.

Image noise and denoise limit#

Two advanced layers for image-to-image work. An image noise layer injects noise into a region so the model reinvents it even at low denoising strength; a denoise limit layer caps how much a region may change. Both are painted like masks and apply only where painted.

Filters on raster layers#

Any raster layer can be run through the same processors as a control layer (Filter in the layer menu) and, with the Image-to-Image Model filter, through an upscale or restoration model. The result replaces the layer pixels; Cancel keeps the original. This is the quickest way to make a depth map or edge map from a photo you dropped in, then convert the layer to a control layer.