API Reference¶
MLArray Module¶
mlarray.mlarray.MLArray ¶
MLArray(array: Optional[Union[ndarray, str, Path]] = None, spacing: Optional[Union[List, Tuple, ndarray]] = None, origin: Optional[Union[List, Tuple, ndarray]] = None, direction: Optional[Union[List, Tuple, ndarray]] = None, affine: Optional[Union[List, Tuple, ndarray]] = None, meta: Optional[Union[Dict, Meta]] = None, axis_labels: Optional[List[Union[str, AxisLabel]]] = None, copy: Optional[MLArray] = None, patch_size: Optional[Union[int, List, Tuple]] = 'default', chunk_size: Optional[Union[int, List, Tuple]] = None, block_size: Optional[Union[int, List, Tuple]] = None, cparams: Optional[Union[Dict, CParams]] = None, dparams: Optional[Union[Dict, DParams]] = None, compressed: bool = True)
Initializes a MLArray instance.
The MLArray file format (".mla") is a Blosc2-compressed container with standardized metadata support for N-dimensional medical images.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
array
|
Optional[Union[ndarray, str, Path]]
|
Input data or file path. Use a numpy ndarray for in-memory arrays, or a string/Path to load a ".mla" file. If None, an empty MLArray instance is created. |
None
|
spacing
|
Optional[Union[List, Tuple, ndarray]]
|
Spacing per spatial axis. Provide a list/tuple/ndarray with length equal to the number of spatial dimensions (e.g., [sx, sy, sz]). |
None
|
origin
|
Optional[Union[List, Tuple, ndarray]]
|
Origin per axis. Provide a list/tuple/ndarray with length equal to the number of spatial dimensions. |
None
|
direction
|
Optional[Union[List, Tuple, ndarray]]
|
Direction cosine matrix. Provide a 2D list/tuple/ndarray with shape (ndims, ndims) for spatial dimensions. |
None
|
affine
|
Optional[Union[List, Tuple, ndarray]]
|
Homogeneous affine matrix. Provide a 2D list/tuple/ndarray with shape (spatial_ndims + 1, spatial_ndims + 1). |
None
|
meta
|
Optional[Dict | Meta]
|
Free-form metadata dictionary or Meta
instance. Must be JSON-serializable when saving.
If meta is passed as a Dict, it is internally converted into a
|
None
|
axis_labels
|
Optional[List[Union[str, AxisLabel]]]
|
Per-axis labels or roles. Length must match ndims. If None, the array is treated as purely spatial. |
None
|
copy
|
Optional[MLArray]
|
Another MLArray instance to copy metadata fields from. If provided, its metadata overrides any metadata set via arguments. |
None
|
patch_size
|
Optional[Union[int, List, Tuple]]
|
Patch size hint for
chunk/block optimization. Use |
'default'
|
chunk_size
|
Optional[Union[int, List, Tuple]]
|
Explicit chunk size.
Ignored when |
None
|
block_size
|
Optional[Union[int, List, Tuple]]
|
Explicit block size.
Ignored when |
None
|
cparams
|
Optional[Union[Dict, CParams]]
|
Blosc2 compression parameters used when creating in-memory compressed array data. |
None
|
dparams
|
Optional[Union[Dict, DParams]]
|
Blosc2 decompression parameters used for chunk access. |
None
|
affine
property
writable
¶
affine: ndarray
Computes the affine transformation matrix for the image.
Returns:
| Name | Type | Description |
|---|---|---|
list |
ndarray
|
Affine matrix with shape (ndims + 1, ndims + 1), or None if no array is loaded. |
direction
property
writable
¶
direction
Returns the image direction.
Returns:
| Name | Type | Description |
|---|---|---|
list |
Direction cosine matrix with shape (ndims, ndims). |
dtype
property
¶
dtype
Returns the dtype of the array.
Returns:
| Type | Description |
|---|---|
|
np.dtype: Dtype of the underlying array, or None if no array is loaded. |
ndim
property
¶
ndim: int
Returns the number of spatial and non-spatial dimensions of the array.
Returns:
| Name | Type | Description |
|---|---|---|
int |
int
|
Number of dimensions, or None if no array is loaded. |
origin
property
writable
¶
origin
Returns the image origin.
Returns:
| Name | Type | Description |
|---|---|---|
list |
Origin per spatial axis with length equal to the number of |
|
|
spatial dimensions. |
rotation
property
¶
rotation
Extracts the rotation matrix from the affine matrix.
Returns:
| Name | Type | Description |
|---|---|---|
list |
Rotation matrix with shape (ndims, ndims), or None if no array is loaded. |
scale
property
¶
scale
Extracts the scaling factors from the affine matrix.
Returns:
| Name | Type | Description |
|---|---|---|
list |
Scaling factors per axis with length equal to the number of spatial dimensions, or None if no array is loaded. |
shape
property
¶
shape
Returns the shape of the array.
Returns:
| Name | Type | Description |
|---|---|---|
tuple |
Shape of the underlying array, or None if no array is loaded. |
shear
property
¶
shear
Computes the shear matrix from the affine matrix.
Returns:
| Name | Type | Description |
|---|---|---|
list |
Shear matrix with shape (ndims, ndims), or None if no array is loaded. |
spacing
property
writable
¶
spacing
Returns the image spacing.
Returns:
| Name | Type | Description |
|---|---|---|
list |
Spacing per spatial axis with length equal to the number of |
|
|
spatial dimensions. |
spatial_ndim
property
¶
spatial_ndim: int
Returns the number of spatial dimensions.
Returns:
| Name | Type | Description |
|---|---|---|
int |
int
|
Number of spatial dimensions, or None if no array is loaded. |
translation
property
¶
translation
Extracts the translation vector from the affine matrix.
Returns:
| Name | Type | Description |
|---|---|---|
list |
Translation vector with length equal to the number of spatial dimensions, or None if no array is loaded. |
arange
classmethod
¶
arange(start: Union[int, float], stop: Optional[Union[int, float]] = None, step: Optional[Union[int, float]] = 1, dtype: dtype = None, shape: Optional[Union[int, List, Tuple, ndarray]] = None, c_order: bool = True, meta: Optional[Union[Dict, Meta]] = None, patch_size: Optional[Union[int, List, Tuple]] = None, chunk_size: Optional[Union[int, List, Tuple]] = None, block_size: Optional[Union[int, List, Tuple]] = None, cparams: Optional[Union[Dict, CParams]] = None, dparams: Optional[Union[Dict, DParams]] = None, compressed: bool = True)
Create an in-memory MLArray with evenly spaced values.
Behavior mirrors :func:blosc2.arange while also applying MLArray
metadata and chunk/block optimization settings.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
start
|
Union[int, float]
|
Start of interval. If |
required |
stop
|
Optional[Union[int, float]]
|
End of interval (exclusive). |
None
|
step
|
Optional[Union[int, float]]
|
Spacing between values. |
1
|
dtype
|
dtype
|
Output dtype. If None, inferred similarly to
|
None
|
shape
|
Optional[Union[int, List, Tuple, ndarray]]
|
Target output shape. If None, shape is inferred from start/stop/step. |
None
|
c_order
|
bool
|
Store in C order (row-major) if True. |
True
|
meta
|
Optional[Union[Dict, Meta]]
|
Optional metadata attached to
the created |
None
|
patch_size
|
Optional[Union[int, List, Tuple]]
|
Patch size hint for chunk/block optimization. Defaults to None for this method. |
None
|
chunk_size
|
Optional[Union[int, List, Tuple]]
|
Explicit chunk size. |
None
|
block_size
|
Optional[Union[int, List, Tuple]]
|
Explicit block size. |
None
|
cparams
|
Optional[Union[Dict, CParams]]
|
Blosc2 compression parameters. |
None
|
dparams
|
Optional[Union[Dict, DParams]]
|
Blosc2 decompression parameters. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
MLArray |
A newly created in-memory MLArray instance. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
asarray
classmethod
¶
asarray(array: Union[ndarray], meta: Optional[Union[Dict, Meta]] = None, patch_size: Optional[Union[int, List, Tuple]] = 'default', chunk_size: Optional[Union[int, List, Tuple]] = None, block_size: Optional[Union[int, List, Tuple]] = None, cparams: Optional[Union[Dict, CParams]] = None, dparams: Optional[Union[Dict, DParams]] = None, compressed: bool = True)
Convert a NumPy array into an in-memory Blosc2-backed MLArray.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
array
|
Union[ndarray]
|
Input array to convert to MLArray. |
required |
meta
|
Optional[Union[Dict, Meta]]
|
Optional metadata attached to
the created |
None
|
patch_size
|
Optional[Union[int, List, Tuple]]
|
Patch size hint for chunk/block optimization. Provide an int for isotropic sizes or a list/tuple with length equal to the number of dimensions. Use "default" to use the default patch size of 192. |
'default'
|
chunk_size
|
Optional[Union[int, List, Tuple]]
|
Explicit chunk size. Provide an int or a tuple/list with length equal to the number of dimensions, or None to let Blosc2 decide. Ignored when patch_size is not None. |
None
|
block_size
|
Optional[Union[int, List, Tuple]]
|
Explicit block size. Provide an int or a tuple/list with length equal to the number of dimensions, or None to let Blosc2 decide. Ignored when patch_size is not None. |
None
|
cparams
|
Optional[Union[Dict, CParams]]
|
Blosc2
compression parameters used when creating the in-memory Blosc2
container (for example codec, compression level, and filters).
Controls how data is compressed when stored. If None, defaults
to |
None
|
dparams
|
Optional[Union[Dict, DParams]]
|
Blosc2
decompression parameters used when
accessing compressed chunks (for example number of
decompression threads). Controls runtime decompression behavior.
If None, defaults to |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
MLArray |
A newly created MLArray instance. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If |
ValueError
|
If |
RuntimeError
|
If patch/chunk/block arguments are inconsistent. |
NotImplementedError
|
If automatic patch optimization is not implemented for the provided dimensionality. |
close ¶
close()
Flush metadata and close the underlying store.
After closing, the MLArray instance no longer has an attached array.
comp_blosc2_params
classmethod
¶
comp_blosc2_params(image_size: Union[Tuple[int, int], Tuple[int, int, int], Tuple[int, int, int, int]], patch_size: Union[Tuple[int, int], Tuple[int, int, int]], spatial_axis_mask: Optional[list[bool]] = None, bytes_per_pixel: int = 4, l1_cache_size_per_core_in_bytes: int = 32768, l3_cache_size_per_core_in_bytes: int = 1441792, safety_factor: float = 0.8)
Computes a recommended block and chunk size for saving arrays with Blosc v2.
Blosc2 NDIM documentation: "Having a second partition allows for greater flexibility in fitting different partitions to different CPU cache levels. Typically, the first partition (also known as chunks) should be sized to fit within the L3 cache, while the second partition (also known as blocks) should be sized to fit within the L2 or L1 caches, depending on whether the priority is compression ratio or speed." (Source: https://www.blosc.org/posts/blosc2-ndim-intro/)
Our approach is not fully optimized for this yet. Currently, we aim to fit the uncompressed block within the L1 cache, accepting that it might occasionally spill over into L2, which we consider acceptable.
Note: This configuration is specifically optimized for nnU-Net data loading, where each read operation is performed by a single core, so multi-threading is not an option.
The default cache values are based on an older Intel 4110 CPU with 32KB L1, 128KB L2, and 1408KB L3 cache per core. We haven't further optimized for modern CPUs with larger caches, as our data must still be compatible with the older systems.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
image_size
|
Union[Tuple[int, int], Tuple[int, int, int], Tuple[int, int, int, int]]
|
Image shape. Use a 2D, 3D, or 4D size; 2D/3D inputs are internally expanded to 4D (with non-spatial axes first). |
required |
patch_size
|
Union[Tuple[int, int], Tuple[int, int, int]]
|
Patch size for spatial dimensions. Use a 2-tuple (x, y) or 3-tuple (x, y, z). |
required |
spatial_axis_mask
|
Optional[list[bool]]
|
Mask indicating for every axis whether it is spatial or not. |
None
|
bytes_per_pixel
|
int
|
Number of bytes per element. Defaults to 4 for float32. |
4
|
l1_cache_size_per_core_in_bytes
|
int
|
L1 cache per core in bytes. |
32768
|
l3_cache_size_per_core_in_bytes
|
int
|
L3 cache per core in bytes. |
1441792
|
safety_factor
|
float
|
Safety factor to avoid filling caches. |
0.8
|
Returns:
| Type | Description |
|---|---|
|
Tuple[List[int], List[int]]: Recommended chunk size and block size. |
compress ¶
compress(patch_size: Optional[Union[int, List, Tuple]] = 'default', chunk_size: Optional[Union[int, List, Tuple]] = None, block_size: Optional[Union[int, List, Tuple]] = None, cparams: Optional[Union[Dict, CParams]] = None, dparams: Optional[Union[Dict, DParams]] = None) -> None
Compress this MLArray in-place using the Blosc2 backend.
The instance is detached from any previously opened file state and
becomes an in-memory object. Persist changes by calling save().
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
patch_size
|
Optional[Union[int, List, Tuple]]
|
Patch size hint for
chunk/block optimization. Use |
'default'
|
chunk_size
|
Optional[Union[int, List, Tuple]]
|
Explicit chunk size.
Ignored when |
None
|
block_size
|
Optional[Union[int, List, Tuple]]
|
Explicit block size.
Ignored when |
None
|
cparams
|
Optional[Union[Dict, CParams]]
|
Blosc2 compression parameters. |
None
|
dparams
|
Optional[Union[Dict, DParams]]
|
Blosc2 decompression parameters. |
None
|
Raises:
| Type | Description |
|---|---|
TypeError
|
If no array data is loaded. |
create
classmethod
¶
create(filepath: Union[str, Path], shape: Union[List, Tuple, ndarray], dtype: dtype, meta: Optional[Union[Dict, Meta]] = None, mode: str = 'w', mmap_mode: str = 'w+', patch_size: Optional[Union[int, List, Tuple]] = 'default', chunk_size: Optional[Union[int, List, Tuple]] = None, block_size: Optional[Union[int, List, Tuple]] = None, cparams: Optional[Union[Dict, CParams]] = None, dparams: Optional[Union[Dict, DParams]] = None)
Create a new MLArray file with memory mapping.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filepath
|
Union[str, Path]
|
Target file path. Must end with ".mla". |
required |
shape
|
Union[List, Tuple, ndarray]
|
Shape of the array to create. If provided, a new file is created. Length must match the full array dimensionality (including non-spatial axes if present). |
required |
dtype
|
dtype
|
Numpy dtype for a newly created array. |
required |
meta
|
Optional[Union[Dict, Meta]]
|
Optional metadata attached to
the created |
None
|
mode
|
str
|
Controls storage open/creation permissions when using standard Blosc2 I/O (read-only, read/write, overwrite). Does not affect lazy loading or decompression; data is accessed and decompressed on demand by Blosc2. Must be 'w' (default). Leave at default if unsure. |
'w'
|
mmap_mode
|
str
|
Controls access via OS-level memory mapping of the compressed data, including read/write permissions.
Changes how bytes are fetched from disk (paging rather than explicit reads), while chunks are still decompressed on demand by Blosc2.
Overrides |
'w+'
|
patch_size
|
Optional[Union[int, List, Tuple]]
|
Patch size hint for chunk/block optimization. Provide an int for isotropic sizes or a list/tuple with length equal to the number of spatial dimensions. Use "default" to use the default patch size of 192. |
'default'
|
chunk_size
|
Optional[Union[int, List, Tuple]]
|
Explicit chunk size.
Provide an int or tuple/list with length equal to the array
dimensions. Ignored when |
None
|
block_size
|
Optional[Union[int, List, Tuple]]
|
Explicit block size.
Provide an int or tuple/list with length equal to the array
dimensions. Ignored when |
None
|
cparams
|
Optional[Union[Dict, CParams]]
|
Blosc2
compression parameters used when
creating/writing array data (for example codec, compression
level, and filters). Controls how data is compressed when
stored. If None, defaults to |
None
|
dparams
|
Optional[Union[Dict, DParams]]
|
Blosc2
decompression parameters used when
reading/accessing compressed chunks (for example number of
decompression threads). Controls runtime decompression behavior.
If None, defaults to |
None
|
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If the file extension is invalid or if mode/mmap_mode is invalid for creation. |
decompress ¶
decompress() -> None
Decompress this MLArray in-place to a NumPy-backed array.
The instance is detached from any previously opened file state and
becomes an in-memory object. Persist changes by calling save().
meta.blosc2 metadata is cleared.
Raises:
| Type | Description |
|---|---|
TypeError
|
If no array data is loaded. |
empty
classmethod
¶
empty(shape: Union[int, List, Tuple, ndarray], dtype: dtype, meta: Optional[Union[Dict, Meta]] = None, patch_size: Optional[Union[int, List, Tuple]] = 'default', chunk_size: Optional[Union[int, List, Tuple]] = None, block_size: Optional[Union[int, List, Tuple]] = None, cparams: Optional[Union[Dict, CParams]] = None, dparams: Optional[Union[Dict, DParams]] = None, compressed: bool = True)
Create an in-memory MLArray with uninitialized values.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
shape
|
Union[int, List, Tuple, ndarray]
|
Shape of the output array. |
required |
dtype
|
dtype
|
Numpy dtype for the output array. |
required |
meta
|
Optional[Union[Dict, Meta]]
|
Optional metadata attached to
the created |
None
|
patch_size
|
Optional[Union[int, List, Tuple]]
|
Patch size hint for
chunk/block optimization. Use |
'default'
|
chunk_size
|
Optional[Union[int, List, Tuple]]
|
Explicit chunk size.
Ignored when |
None
|
block_size
|
Optional[Union[int, List, Tuple]]
|
Explicit block size.
Ignored when |
None
|
cparams
|
Optional[Union[Dict, CParams]]
|
Blosc2 compression parameters used when writing chunks. |
None
|
dparams
|
Optional[Union[Dict, DParams]]
|
Blosc2 decompression parameters used when reading chunks. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
MLArray |
A newly created in-memory MLArray instance. |
empty_like
classmethod
¶
empty_like(x, dtype: dtype = None, meta: Optional[Union[Dict, Meta]] = None, patch_size: Optional[Union[int, List, Tuple]] = 'default', chunk_size: Optional[Union[int, List, Tuple]] = None, block_size: Optional[Union[int, List, Tuple]] = None, cparams: Optional[Union[Dict, CParams]] = None, dparams: Optional[Union[Dict, DParams]] = None, compressed: bool = True)
Create an in-memory MLArray with the same shape as x.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x
|
Source object. Can be an |
required | |
dtype
|
dtype
|
Output dtype. If None, inferred from |
None
|
meta
|
Optional[Union[Dict, Meta]]
|
Optional metadata attached to
the created |
None
|
patch_size
|
Optional[Union[int, List, Tuple]]
|
Patch size hint for chunk/block optimization. |
'default'
|
chunk_size
|
Optional[Union[int, List, Tuple]]
|
Explicit chunk size. |
None
|
block_size
|
Optional[Union[int, List, Tuple]]
|
Explicit block size. |
None
|
cparams
|
Optional[Union[Dict, CParams]]
|
Blosc2 compression parameters. |
None
|
dparams
|
Optional[Union[Dict, DParams]]
|
Blosc2 decompression parameters. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
MLArray |
A newly created in-memory MLArray instance. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If |
full
classmethod
¶
full(shape: Union[int, List, Tuple, ndarray], fill_value: Union[bytes, int, float, bool], dtype: dtype = None, meta: Optional[Union[Dict, Meta]] = None, patch_size: Optional[Union[int, List, Tuple]] = 'default', chunk_size: Optional[Union[int, List, Tuple]] = None, block_size: Optional[Union[int, List, Tuple]] = None, cparams: Optional[Union[Dict, CParams]] = None, dparams: Optional[Union[Dict, DParams]] = None, compressed: bool = True)
Create an in-memory MLArray filled with fill_value.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
shape
|
Union[int, List, Tuple, ndarray]
|
Shape of the output array. |
required |
fill_value
|
Union[bytes, int, float, bool]
|
Fill value used for all elements in the output. |
required |
dtype
|
dtype
|
Numpy dtype for the output array. If None, inferred
from |
None
|
meta
|
Optional[Union[Dict, Meta]]
|
Optional metadata attached to
the created |
None
|
patch_size
|
Optional[Union[int, List, Tuple]]
|
Patch size hint for chunk/block optimization. |
'default'
|
chunk_size
|
Optional[Union[int, List, Tuple]]
|
Explicit chunk size.
Ignored when |
None
|
block_size
|
Optional[Union[int, List, Tuple]]
|
Explicit block size.
Ignored when |
None
|
cparams
|
Optional[Union[Dict, CParams]]
|
Blosc2 compression parameters. |
None
|
dparams
|
Optional[Union[Dict, DParams]]
|
Blosc2 decompression parameters. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
MLArray |
A newly created in-memory MLArray instance. |
full_like
classmethod
¶
full_like(x, fill_value: Union[bool, int, float, complex], dtype: dtype = None, meta: Optional[Union[Dict, Meta]] = None, patch_size: Optional[Union[int, List, Tuple]] = 'default', chunk_size: Optional[Union[int, List, Tuple]] = None, block_size: Optional[Union[int, List, Tuple]] = None, cparams: Optional[Union[Dict, CParams]] = None, dparams: Optional[Union[Dict, DParams]] = None, compressed: bool = True)
Create an in-memory MLArray filled with fill_value and shape of x.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x
|
Source object. Can be an |
required | |
fill_value
|
Union[bool, int, float, complex]
|
Fill value used for all elements in the output. |
required |
dtype
|
dtype
|
Output dtype. If None, inferred from |
None
|
meta
|
Optional[Union[Dict, Meta]]
|
Optional metadata attached to
the created |
None
|
patch_size
|
Optional[Union[int, List, Tuple]]
|
Patch size hint for chunk/block optimization. |
'default'
|
chunk_size
|
Optional[Union[int, List, Tuple]]
|
Explicit chunk size. |
None
|
block_size
|
Optional[Union[int, List, Tuple]]
|
Explicit block size. |
None
|
cparams
|
Optional[Union[Dict, CParams]]
|
Blosc2 compression parameters. |
None
|
dparams
|
Optional[Union[Dict, DParams]]
|
Blosc2 decompression parameters. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
MLArray |
A newly created in-memory MLArray instance. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If |
linspace
classmethod
¶
linspace(start: Union[int, float, complex], stop: Union[int, float, complex], num: Optional[int] = None, dtype: dtype = None, endpoint: bool = True, shape: Optional[Union[int, List, Tuple, ndarray]] = None, c_order: bool = True, meta: Optional[Union[Dict, Meta]] = None, patch_size: Optional[Union[int, List, Tuple]] = None, chunk_size: Optional[Union[int, List, Tuple]] = None, block_size: Optional[Union[int, List, Tuple]] = None, cparams: Optional[Union[Dict, CParams]] = None, dparams: Optional[Union[Dict, DParams]] = None, compressed: bool = True)
Create an in-memory MLArray with evenly spaced samples.
Behavior mirrors :func:blosc2.linspace while also applying MLArray
metadata and chunk/block optimization settings.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
start
|
Union[int, float, complex]
|
Start value of the sequence. |
required |
stop
|
Union[int, float, complex]
|
End value of the sequence. |
required |
num
|
Optional[int]
|
Number of samples. Required when |
None
|
dtype
|
dtype
|
Output dtype. If None, inferred similarly to
|
None
|
endpoint
|
bool
|
Whether |
True
|
shape
|
Optional[Union[int, List, Tuple, ndarray]]
|
Target output
shape. If None, inferred from |
None
|
c_order
|
bool
|
Store in C order (row-major) if True. |
True
|
meta
|
Optional[Union[Dict, Meta]]
|
Optional metadata attached to
the created |
None
|
patch_size
|
Optional[Union[int, List, Tuple]]
|
Patch size hint for chunk/block optimization. Defaults to None for this method. |
None
|
chunk_size
|
Optional[Union[int, List, Tuple]]
|
Explicit chunk size. |
None
|
block_size
|
Optional[Union[int, List, Tuple]]
|
Explicit block size. |
None
|
cparams
|
Optional[Union[Dict, CParams]]
|
Blosc2 compression parameters. |
None
|
dparams
|
Optional[Union[Dict, DParams]]
|
Blosc2 decompression parameters. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
MLArray |
A newly created in-memory MLArray instance. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If neither |
load
classmethod
¶
load(filepath: Union[str, Path], dparams: Optional[Union[Dict, DParams]] = None, compressed: bool = True)
Load a MLArray file as a whole without memory mapping.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filepath
|
Union[str, Path]
|
Path to the MLArray file to be loaded. The filepath needs to have the extension ".mla". |
required |
dparams
|
Optional[Union[Dict, DParams]]
|
Blosc2
decompression parameters used when loading/accessing compressed
chunks. If None, defaults to |
None
|
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If the file extension is not ".mla". |
ones
classmethod
¶
ones(shape: Union[int, List, Tuple, ndarray], dtype: dtype = None, meta: Optional[Union[Dict, Meta]] = None, patch_size: Optional[Union[int, List, Tuple]] = 'default', chunk_size: Optional[Union[int, List, Tuple]] = None, block_size: Optional[Union[int, List, Tuple]] = None, cparams: Optional[Union[Dict, CParams]] = None, dparams: Optional[Union[Dict, DParams]] = None, compressed: bool = True)
Create an in-memory MLArray filled with ones.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
shape
|
Union[int, List, Tuple, ndarray]
|
Shape of the output array. |
required |
dtype
|
dtype
|
Numpy dtype for the output array. If None, uses
|
None
|
meta
|
Optional[Union[Dict, Meta]]
|
Optional metadata attached to
the created |
None
|
patch_size
|
Optional[Union[int, List, Tuple]]
|
Patch size hint for chunk/block optimization. |
'default'
|
chunk_size
|
Optional[Union[int, List, Tuple]]
|
Explicit chunk size.
Ignored when |
None
|
block_size
|
Optional[Union[int, List, Tuple]]
|
Explicit block size.
Ignored when |
None
|
cparams
|
Optional[Union[Dict, CParams]]
|
Blosc2 compression parameters. |
None
|
dparams
|
Optional[Union[Dict, DParams]]
|
Blosc2 decompression parameters. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
MLArray |
A newly created in-memory MLArray instance. |
ones_like
classmethod
¶
ones_like(x, dtype: dtype = None, meta: Optional[Union[Dict, Meta]] = None, patch_size: Optional[Union[int, List, Tuple]] = 'default', chunk_size: Optional[Union[int, List, Tuple]] = None, block_size: Optional[Union[int, List, Tuple]] = None, cparams: Optional[Union[Dict, CParams]] = None, dparams: Optional[Union[Dict, DParams]] = None, compressed: bool = True)
Create an in-memory MLArray of ones with the same shape as x.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x
|
Source object. Can be an |
required | |
dtype
|
dtype
|
Output dtype. If None, inferred from |
None
|
meta
|
Optional[Union[Dict, Meta]]
|
Optional metadata attached to
the created |
None
|
patch_size
|
Optional[Union[int, List, Tuple]]
|
Patch size hint for chunk/block optimization. |
'default'
|
chunk_size
|
Optional[Union[int, List, Tuple]]
|
Explicit chunk size. |
None
|
block_size
|
Optional[Union[int, List, Tuple]]
|
Explicit block size. |
None
|
cparams
|
Optional[Union[Dict, CParams]]
|
Blosc2 compression parameters. |
None
|
dparams
|
Optional[Union[Dict, DParams]]
|
Blosc2 decompression parameters. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
MLArray |
A newly created in-memory MLArray instance. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If |
open
classmethod
¶
open(filepath: Union[str, Path], mode: str = 'r', mmap_mode: str = 'r', dparams: Optional[Union[Dict, DParams]] = None)
Open an existing MLArray file with memory mapping.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filepath
|
Union[str, Path]
|
Target file path. Must end with ".mla". |
required |
mode
|
str
|
Controls storage open/creation permissions when using standard Blosc2 I/O (read-only, read/write, overwrite). Does not affect lazy loading or decompression; data is accessed and decompressed on demand by Blosc2. Must be either 'r' (default) or 'a'. Leave at default if unsure. |
'r'
|
mmap_mode
|
str
|
Controls access via OS-level memory mapping of the compressed data, including read/write permissions.
Changes how bytes are fetched from disk (paging rather than explicit reads), while chunks are still decompressed on demand by Blosc2.
Overrides |
'r'
|
dparams
|
Optional[Union[Dict, DParams]]
|
Blosc2
decompression parameters used when
reading/accessing compressed chunks (for example number of
decompression threads). Controls runtime decompression behavior.
If None, defaults to |
None
|
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If the file extension is invalid or if mode/mmap_mode is invalid for opening. |
save ¶
save(filepath: Union[str, Path])
Save the array to a MLArray file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filepath
|
Union[str, Path]
|
Path to save the file. Must end with ".mla". |
required |
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If the file extension is not ".mla". |
to_numpy ¶
to_numpy()
Return the underlying data as a NumPy array.
Returns:
| Type | Description |
|---|---|
|
np.ndarray: A NumPy view or copy of the stored array data. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If no array data is loaded. |
zeros
classmethod
¶
zeros(shape: Union[int, List, Tuple, ndarray], dtype: dtype, meta: Optional[Union[Dict, Meta]] = None, patch_size: Optional[Union[int, List, Tuple]] = 'default', chunk_size: Optional[Union[int, List, Tuple]] = None, block_size: Optional[Union[int, List, Tuple]] = None, cparams: Optional[Union[Dict, CParams]] = None, dparams: Optional[Union[Dict, DParams]] = None, compressed: bool = True)
Create an in-memory MLArray filled with zeros.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
shape
|
Union[int, List, Tuple, ndarray]
|
Shape of the output array. |
required |
dtype
|
dtype
|
Numpy dtype for the output array. |
required |
meta
|
Optional[Union[Dict, Meta]]
|
Optional metadata attached to
the created |
None
|
patch_size
|
Optional[Union[int, List, Tuple]]
|
Patch size hint for chunk/block optimization. |
'default'
|
chunk_size
|
Optional[Union[int, List, Tuple]]
|
Explicit chunk size.
Ignored when |
None
|
block_size
|
Optional[Union[int, List, Tuple]]
|
Explicit block size.
Ignored when |
None
|
cparams
|
Optional[Union[Dict, CParams]]
|
Blosc2 compression parameters. |
None
|
dparams
|
Optional[Union[Dict, DParams]]
|
Blosc2 decompression parameters. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
MLArray |
A newly created in-memory MLArray instance. |
zeros_like
classmethod
¶
zeros_like(x, dtype: dtype = None, meta: Optional[Union[Dict, Meta]] = None, patch_size: Optional[Union[int, List, Tuple]] = 'default', chunk_size: Optional[Union[int, List, Tuple]] = None, block_size: Optional[Union[int, List, Tuple]] = None, cparams: Optional[Union[Dict, CParams]] = None, dparams: Optional[Union[Dict, DParams]] = None, compressed: bool = True)
Create an in-memory MLArray of zeros with the same shape as x.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x
|
Source object. Can be an |
required | |
dtype
|
dtype
|
Output dtype. If None, inferred from |
None
|
meta
|
Optional[Union[Dict, Meta]]
|
Optional metadata attached to
the created |
None
|
patch_size
|
Optional[Union[int, List, Tuple]]
|
Patch size hint for chunk/block optimization. |
'default'
|
chunk_size
|
Optional[Union[int, List, Tuple]]
|
Explicit chunk size. |
None
|
block_size
|
Optional[Union[int, List, Tuple]]
|
Explicit block size. |
None
|
cparams
|
Optional[Union[Dict, CParams]]
|
Blosc2 compression parameters. |
None
|
dparams
|
Optional[Union[Dict, DParams]]
|
Blosc2 decompression parameters. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
MLArray |
A newly created in-memory MLArray instance. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If |
Metadata Module¶
mlarray.meta.BaseMeta
dataclass
¶
BaseMeta()
Base class for metadata containers.
Subclasses should implement _validate_and_cast to coerce and validate fields after initialization or mutation.
copy_from ¶
copy_from(other: T, *, overwrite: bool = False) -> None
Copy fields from another instance of the same class.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
other
|
T
|
Source instance. |
required |
overwrite
|
bool
|
When True, overwrite all fields. When False, only fill destination fields that are "unset" (None or empty containers). Nested BaseMeta fields are merged recursively unless the entire destination sub-meta is default, in which case it is replaced. |
False
|
Raises:
| Type | Description |
|---|---|
TypeError
|
If other is not the same class as self. |
ensure
classmethod
¶
ensure(x: Any) -> T
Coerce x into an instance of cls.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x
|
Any
|
None, an instance of cls, or a mapping of fields. |
required |
Returns:
| Type | Description |
|---|---|
T
|
An instance of cls. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If x is not None, cls, or a mapping. |
from_mapping
classmethod
¶
from_mapping(d: Mapping[str, Any]) -> T
Construct an instance from a mapping.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
d
|
Mapping[str, Any]
|
Input mapping matching dataclass field names. |
required |
Returns:
| Type | Description |
|---|---|
T
|
A new instance of cls. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If d is not a Mapping. |
KeyError
|
If unknown keys are present. |
to_mapping ¶
to_mapping(*, include_none: bool = True) -> dict[str, Any]
Serialize to a mapping, recursively expanding nested BaseMeta.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
include_none
|
bool
|
Include fields with None values when True. |
True
|
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
A dict of field names to serialized values. |
to_plain ¶
to_plain(*, include_none: bool = False) -> Any
Convert to plain Python objects recursively.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
include_none
|
bool
|
Include fields with None values when True. |
False
|
Returns:
| Type | Description |
|---|---|
Any
|
A dict of field values, with nested BaseMeta expanded. SingleKeyBaseMeta |
Any
|
overrides this to return its wrapped value. |
mlarray.meta.SingleKeyBaseMeta
dataclass
¶
SingleKeyBaseMeta()
Bases: BaseMeta
BaseMeta subclass that wraps a single field as a raw value.
ensure
classmethod
¶
ensure(x: Any) -> SK
Coerce input into an instance of cls.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x
|
Any
|
None, instance of cls, mapping, or raw value. |
required |
Returns:
| Type | Description |
|---|---|
SK
|
An instance of cls. |
from_mapping
classmethod
¶
from_mapping(d: Any) -> SK
Construct from either schema-shaped mapping or raw value.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
d
|
Any
|
None, mapping, or raw value. |
required |
Returns:
| Type | Description |
|---|---|
SK
|
A new instance of cls. |
to_mapping ¶
to_mapping(*, include_none: bool = True) -> dict[str, Any]
Serialize to a mapping with the single key.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
include_none
|
bool
|
Include the key when the value is None. |
True
|
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
A dict with the single field name as the key, or an empty dict. |
to_plain ¶
to_plain(*, include_none: bool = False) -> Any
Return the wrapped value for plain output.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
include_none
|
bool
|
Return None when the value is None. |
False
|
Returns:
| Type | Description |
|---|---|
Any
|
The wrapped value or None. |
mlarray.meta.Meta
dataclass
¶
Meta(source: 'MetaSource' = (lambda: MetaSource())(), extra: 'MetaExtra' = (lambda: MetaExtra())(), spatial: 'MetaSpatial' = (lambda: MetaSpatial())(), stats: 'MetaStatistics' = (lambda: MetaStatistics())(), bbox: 'MetaBbox' = (lambda: MetaBbox())(), is_seg: 'MetaIsSeg' = (lambda: MetaIsSeg())(), blosc2: 'MetaBlosc2' = (lambda: MetaBlosc2())(), _has_array: 'MetaHasArray' = (lambda: MetaHasArray())(), _image_meta_format: 'MetaImageFormat' = (lambda: MetaImageFormat())(), _mlarray_version: 'MetaVersion' = (lambda: MetaVersion())())
Bases: BaseMeta
Top-level metadata container for mlarray.
Attributes:
| Name | Type | Description |
|---|---|---|
source |
'MetaSource'
|
Source metadata from the original image source (JSON-serializable dict). |
extra |
'MetaExtra'
|
Additional metadata (JSON-serializable dict). |
spatial |
'MetaSpatial'
|
Spatial metadata (spacing, origin, direction, affine, shape). |
stats |
'MetaStatistics'
|
Summary statistics. |
bbox |
'MetaBbox'
|
Bounding boxes. |
is_seg |
'MetaIsSeg'
|
Segmentation flag. |
blosc2 |
'MetaBlosc2'
|
Blosc2 chunking/tiling metadata. |
_has_array |
'MetaHasArray'
|
Payload presence flag. |
_image_meta_format |
'MetaImageFormat'
|
Image metadata format identifier. |
_mlarray_version |
'MetaVersion'
|
Version string for mlarray. |
to_plain ¶
to_plain(*, include_none: bool = False) -> Any
Convert to plain values, suppressing default sub-metas.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
include_none
|
bool
|
Include None values when True. |
False
|
Returns:
| Type | Description |
|---|---|
Any
|
A dict of field values where default child metas are represented |
Any
|
as None and optionally filtered out. |
mlarray.meta.MetaSource
dataclass
¶
MetaSource(data: dict[str, Any] = dict())
Bases: SingleKeyBaseMeta
Source metadata from the original image source stored as JSON-serializable dict.
Attributes:
| Name | Type | Description |
|---|---|---|
data |
dict[str, Any]
|
Arbitrary JSON-serializable metadata. |
mlarray.meta.MetaExtra
dataclass
¶
MetaExtra(data: dict[str, Any] = dict())
Bases: SingleKeyBaseMeta
Generic extra metadata stored as JSON-serializable dict.
Attributes:
| Name | Type | Description |
|---|---|---|
data |
dict[str, Any]
|
Arbitrary JSON-serializable metadata. |
mlarray.meta.MetaSpatial
dataclass
¶
MetaSpatial(spacing: Optional[list[Union[int, float]]] = None, origin: Optional[list[Union[int, float]]] = None, direction: Optional[list[list[Union[int, float]]]] = None, affine: Optional[list[list[Union[int, float]]]] = None, shape: Optional[list[int]] = None, axis_labels: Optional[list[Union[str, AxisLabel]]] = None, axis_units: Optional[list[str]] = None, _num_spatial_axes: Optional[int] = None, _num_non_spatial_axes: Optional[int] = None)
Bases: BaseMeta
Spatial metadata describing geometry and layout.
Attributes:
| Name | Type | Description |
|---|---|---|
spacing |
Optional[list[Union[int, float]]]
|
Per-dimension spacing values. Length must match ndims. |
origin |
Optional[list[Union[int, float]]]
|
Per-dimension origin values. Length must match ndims. |
direction |
Optional[list[list[Union[int, float]]]]
|
Direction cosine matrix of shape [ndims, ndims]. |
affine |
Optional[list[list[Union[int, float]]]]
|
Homogeneous affine matrix of shape [ndims + 1, ndims + 1]. |
shape |
Optional[list[int]]
|
Array shape. Length must match (spatial + non-spatial) ndims. |
axis_labels |
Optional[list[Union[str, AxisLabel]]]
|
Per-axis labels or roles. Length must match ndims. |
axis_units |
Optional[list[str]]
|
Per-axis units. Length must match ndims. |
_num_spatial_axes |
Optional[int]
|
Cached count of spatial axes derived from axis_labels. |
_num_non_spatial_axes |
Optional[int]
|
Cached count of non-spatial axes derived from axis_labels. |
mlarray.meta.AxisLabelEnum ¶
Bases: str, Enum
Axis label/role identifiers used for spatial metadata.
Attributes:
| Name | Type | Description |
|---|---|---|
spatial |
Generic spatial axis (used when no axis-specific label applies). |
|
spatial_x |
Spatial axis representing X. |
|
spatial_y |
Spatial axis representing Y. |
|
spatial_z |
Spatial axis representing Z. |
|
non_spatial |
Generic non-spatial axis. |
|
channel |
Channel axis (e.g., color channels or feature maps). |
|
temporal |
Time axis. |
|
continuous |
Continuous-valued axis (non-spatial). |
|
components |
Component axis (e.g., vector components). |
mlarray.meta.MetaStatistics
dataclass
¶
MetaStatistics(min: Optional[float] = None, max: Optional[float] = None, mean: Optional[float] = None, median: Optional[float] = None, std: Optional[float] = None, percentile_min: Optional[float] = None, percentile_max: Optional[float] = None, percentile_mean: Optional[float] = None, percentile_median: Optional[float] = None, percentile_std: Optional[float] = None, percentile_min_key: Optional[float] = None, percentile_max_key: Optional[float] = None)
Bases: BaseMeta
Numeric summary statistics for an array.
Attributes:
| Name | Type | Description |
|---|---|---|
min |
Optional[float]
|
Minimum value. |
max |
Optional[float]
|
Maximum value. |
mean |
Optional[float]
|
Mean value. |
median |
Optional[float]
|
Median value. |
std |
Optional[float]
|
Standard deviation. |
percentile_min |
Optional[float]
|
Minimum percentile value. |
percentile_max |
Optional[float]
|
Maximum percentile value. |
percentile_mean |
Optional[float]
|
Mean percentile value. |
percentile_median |
Optional[float]
|
Median percentile value. |
percentile_std |
Optional[float]
|
Standard deviation of percentile values. |
percentile_min_key |
Optional[float]
|
Minimum percentile key used to determine percentile_min (for example 0.05). |
percentile_max_key |
Optional[float]
|
Maximum percentile key used to determine percentile_max (for example 0.95). |
mlarray.meta.MetaBbox
dataclass
¶
MetaBbox(bboxes: Optional[list[list[list[Union[int, float]]]]] = None, scores: Optional[list[Union[int, float]]] = None, labels: Optional[list[Union[str, int, float]]] = None)
Bases: BaseMeta
Bounding box metadata with optional scores and labels.
Attributes:
| Name | Type | Description |
|---|---|---|
bboxes |
Optional[list[list[list[Union[int, float]]]]]
|
List of bounding boxes with shape [n_boxes, ndims, 2], where each inner pair is [min, max] for a dimension. Values must be ints or floats. |
scores |
Optional[list[Union[int, float]]]
|
Optional confidence scores aligned with bboxes (ints or floats). |
labels |
Optional[list[Union[str, int, float]]]
|
Optional labels aligned with bboxes. Each label may be a string, int, or float. |
mlarray.meta.MetaIsSeg
dataclass
¶
MetaIsSeg(is_seg: Optional[bool] = None)
Bases: SingleKeyBaseMeta
Flag indicating whether the array is a segmentation mask.
Attributes:
| Name | Type | Description |
|---|---|---|
is_seg |
Optional[bool]
|
True/False when known, None when unknown. |
mlarray.meta.MetaBlosc2
dataclass
¶
MetaBlosc2(chunk_size: Optional[list] = None, block_size: Optional[list] = None, patch_size: Optional[list] = None, cparams: Optional[dict[str, Any]] = None, dparams: Optional[dict[str, Any]] = None)
Bases: BaseMeta
Metadata for Blosc2 tiling and chunking.
Attributes:
| Name | Type | Description |
|---|---|---|
chunk_size |
Optional[list]
|
List of per-dimension chunk sizes. Length must match ndims. |
block_size |
Optional[list]
|
List of per-dimension block sizes. Length must match ndims. |
patch_size |
Optional[list]
|
List of per-dimension patch sizes. Length must match spatial ndims. |
cparams |
Optional[dict[str, Any]]
|
Blosc2 compression parameters as a JSON-serializable dict. |
dparams |
Optional[dict[str, Any]]
|
Blosc2 decompression parameters as a JSON-serializable dict. |
mlarray.meta.MetaHasArray
dataclass
¶
MetaHasArray(has_array: bool = False)
Bases: SingleKeyBaseMeta
Flag indicating whether an array is present.
Attributes:
| Name | Type | Description |
|---|---|---|
has_array |
bool
|
True when array data is present. |
mlarray.meta.MetaImageFormat
dataclass
¶
MetaImageFormat(image_meta_format: Optional[str] = None)
Bases: SingleKeyBaseMeta
String describing the image metadata format.
Attributes:
| Name | Type | Description |
|---|---|---|
image_meta_format |
Optional[str]
|
Format identifier, or None. |
mlarray.meta.MetaVersion
dataclass
¶
MetaVersion(mlarray_version: Optional[str] = None)
Bases: SingleKeyBaseMeta
Version metadata for mlarray.
Attributes:
| Name | Type | Description |
|---|---|---|
mlarray_version |
Optional[str]
|
Version string, or None. |