types¶
common¶
- class TorrentVersion(*values)[source]¶
Version of the bittorrent .torrent file spec
- v1 = 'v1'¶
- v2 = 'v2'¶
- hybrid = 'hybrid'¶
- FileName¶
Placeholder in case specific validation is needed for filenames
alias of
Annotated[str,PlainSerializer(func=_to_bytes, return_type=PydanticUndefined, when_used=always)]
- FilePart¶
Placeholder in case specific validation is needed for filenames
alias of
Annotated[str,PlainSerializer(func=_to_bytes, return_type=PydanticUndefined, when_used=always)]
- class TrackerFields¶
The announce and announce-list
- announce: NotRequired[AnyUrl | str]¶
- pydantic model GenericFileItem[source]¶
File metadata object with file information from both v1 and v2 representations
For use with
Torrent.filesShow JSON schema
{ "title": "GenericFileItem", "description": "File metadata object with file information from both v1 and v2 representations\n\nFor use with :class:`.Torrent.files`", "type": "object", "properties": { "path": { "title": "Path", "type": "string" }, "length": { "minimum": 0, "title": "Length", "type": "integer" }, "attr": { "anyOf": [ { "format": "binary", "type": "string" }, { "type": "null" } ], "default": null, "title": "Attr" }, "pieces root": { "anyOf": [ { "format": "binary", "type": "string" }, { "type": "null" } ], "default": null, "title": "Pieces Root" } }, "additionalProperties": true, "required": [ "path", "length" ] }
- Config:
populate_by_name: bool = True
extra: str = allow
validate_by_alias: bool = True
validate_by_name: bool = True
- Fields:
- field path: Annotated[str, PlainSerializer(func=_to_bytes, return_type=PydanticUndefined, when_used=always)] [Required]¶
- Constraints:
func = <function _to_bytes at 0x70dff2da7380>
return_type = PydanticUndefined
when_used = always
- pydantic model PieceRange[source]¶
Parent model for v1 and v2 piece ranges.
Piece ranges provide some description of paths and byte ranges that correspond to a single verifiable piece and a method for verifying data against them.
Since v1 and v2 data models are substantially different, their sub-models are also quite different, but provide a common interface through this ABC
Show JSON schema
{ "title": "PieceRange", "description": "Parent model for v1 and v2 piece ranges.\n\nPiece ranges provide some description of paths and byte ranges that correspond to a single\nverifiable piece and a method for verifying data against them.\n\nSince v1 and v2 data models are substantially different,\ntheir sub-models are also quite different, but provide a common interface through this ABC", "type": "object", "properties": { "piece_idx": { "title": "Piece Idx", "type": "integer" } }, "required": [ "piece_idx" ] }
- Fields:
- webseed_url(base_url: str, path: str) str[source]¶
Given some base url that is to be used as a webseed url and a path within a torrent file, get the full url that should be requested from the webseed server - leave url unchanged in the case of single file torrents - quote path segments - handle duplicate leading/trailing slashes
serdes¶
Types used only in model serialization and deserialization: AKA types that do not represent concrete types/fields in a torrent file
v1¶
Types used only in v1 (and hybrid) torrents
- V1PieceLength¶
no specification, but “almost always a power of two”, so we validate that.
- Type:
According to BEP 003
alias of
Annotated[int,AfterValidator(func=_power_of_two)]
- pydantic model FileItem[source]¶
Show JSON schema
{ "title": "FileItem", "type": "object", "properties": { "length": { "minimum": 0, "title": "Length", "type": "integer" }, "path": { "items": { "type": "string" }, "title": "Path", "type": "array" }, "attr": { "anyOf": [ { "format": "binary", "type": "string" }, { "type": "null" } ], "default": null, "title": "Attr" } }, "additionalProperties": true, "required": [ "length", "path" ] }
- Config:
populate_by_name: bool = True
extra: str = allow
validate_by_alias: bool = True
validate_by_name: bool = True
- Fields:
- Validators:
strict_padfile_naming»all fields
- field attr: bytes | None = None¶
BEP0047: A variable-length string. When present the characters each represent a file attribute. l = symlink, x = executable, h = hidden, p = padding file. Characters appear in no particular order and unknown characters should be ignored.
- Validated by:
- field path: list[Annotated[str, PlainSerializer(func=_to_bytes, return_type=PydanticUndefined, when_used=always)]] [Required]¶
- Validated by:
- pydantic model FileItemRange[source]¶
A File Item with a byte range, for use with V1PieceRange
Show JSON schema
{ "title": "FileItemRange", "description": "A File Item with a byte range, for use with V1PieceRange", "type": "object", "properties": { "length": { "minimum": 0, "title": "Length", "type": "integer" }, "path": { "items": { "type": "string" }, "title": "Path", "type": "array" }, "attr": { "anyOf": [ { "format": "binary", "type": "string" }, { "type": "null" } ], "default": null, "title": "Attr" }, "range_start": { "title": "Range Start", "type": "integer" }, "range_end": { "title": "Range End", "type": "integer" }, "full_path": { "title": "Full Path", "type": "string" } }, "additionalProperties": true, "required": [ "length", "path", "range_start", "range_end", "full_path" ] }
- Config:
populate_by_name: bool = True
extra: str = allow
validate_by_alias: bool = True
validate_by_name: bool = True
- Fields:
- Validators:
- field full_path: str [Required]¶
Path to be used with webseeds, includes info.name in the case of multifile torrents, so the webseed base can be directly joined with full_path
- Validated by:
- pydantic model V1PieceRange[source]¶
Paths and byte ranges that correspond to a single v1
Show JSON schema
{ "title": "V1PieceRange", "description": "Paths and byte ranges that correspond to a single v1", "type": "object", "properties": { "piece_idx": { "title": "Piece Idx", "type": "integer" }, "ranges": { "items": { "$ref": "#/$defs/FileItemRange" }, "title": "Ranges", "type": "array" }, "piece_hash": { "format": "binary", "maxLength": 20, "minLength": 20, "title": "Piece Hash", "type": "string" } }, "$defs": { "FileItemRange": { "additionalProperties": true, "description": "A File Item with a byte range, for use with V1PieceRange", "properties": { "length": { "minimum": 0, "title": "Length", "type": "integer" }, "path": { "items": { "type": "string" }, "title": "Path", "type": "array" }, "attr": { "anyOf": [ { "format": "binary", "type": "string" }, { "type": "null" } ], "default": null, "title": "Attr" }, "range_start": { "title": "Range Start", "type": "integer" }, "range_end": { "title": "Range End", "type": "integer" }, "full_path": { "title": "Full Path", "type": "string" } }, "required": [ "length", "path", "range_start", "range_end", "full_path" ], "title": "FileItemRange", "type": "object" } }, "required": [ "piece_idx", "ranges", "piece_hash" ] }
- Fields:
- field piece_hash: Annotated[bytes, Len(min_length=20, max_length=20), PlainSerializer(func=_serialize_hash, return_type=PydanticUndefined, when_used=always)] [Required]¶
- Constraints:
min_length = 20
max_length = 20
func = <function _serialize_hash at 0x70dff2d99b20>
return_type = PydanticUndefined
when_used = always
- field ranges: list[FileItemRange] [Required]¶
v2¶
Types used only in v2 (and hybrid) torrents
- V2PieceLength¶
“must be a power of two and at least 16KiB”
- Type:
Per BEP 52
alias of
Annotated[int,AfterValidator(func=_divisible_by_16kib),AfterValidator(func=_power_of_two)]
- pydantic model MerkleTree[source]¶
Representation and computation of v2 merkle trees
A v2 merkle tree is a branching factor 2 tree where each of the leaf nodes is a 16KiB block.
Two layers of the tree are embedded in a torrent file:
the
piece layer: the hashes frompiece length/16KiBlayers from the leaves. or, the layer where each hash corresponds to a chunk of the filepiece lengthlong.the tree root.
Padding is performed in two steps:
For files whose size is not a multiple of
piece length, pad the leaf hashes with zeros (the hashes, not the leaf data, i.e. 32 bytes not 16KiB of zeros) such that there are enough blocks to complete a pieceFor files there the number of pieces does not create a balanced merkle tree, pad the pieces hashes with identical piece hashes each
piece lengthlong s.t. their leaf hashes are all zeros, as above.
These are separated to avoid computing hashes of zero’s unnecessarily.
References
Show JSON schema
{ "title": "MerkleTree", "description": "Representation and computation of v2 merkle trees\n\nA v2 merkle tree is a branching factor 2 tree where each of the leaf nodes is a 16KiB block.\n\nTwo layers of the tree are embedded in a torrent file:\n\n- the ``piece layer``: the hashes from ``piece length/16KiB`` layers from the leaves.\n or, the layer where each hash corresponds to a chunk of the file ``piece length`` long.\n- the tree root.\n\nPadding is performed in two steps:\n\n- For files whose size is not a multiple of ``piece length``,\n pad the *leaf hashes* with zeros\n (the hashes, not the leaf data, i.e. 32 bytes not 16KiB of zeros)\n such that there are enough blocks to complete a piece\n- For files there the number of pieces does not create a balanced merkle tree,\n pad the *pieces hashes* with identical piece hashes each ``piece length`` long\n s.t. their leaf hashes are all zeros, as above.\n\nThese are separated to avoid computing hashes of zero's unnecessarily.\n\nReferences:\n - https://www.bittorrent.org/beps/bep_0052_torrent_creator.py", "type": "object", "properties": { "path": { "format": "path", "title": "Path", "type": "string" }, "piece_length": { "title": "Piece Length", "type": "integer" }, "piece_hashes": { "anyOf": [ { "items": { "format": "binary", "maxLength": 32, "minLength": 32, "type": "string" }, "type": "array" }, { "type": "null" } ], "title": "Piece Hashes" }, "root_hash": { "format": "binary", "title": "Root Hash", "type": "string" }, "leaf_hashes": { "anyOf": [ { "items": { "format": "binary", "maxLength": 32, "minLength": 32, "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Leaf Hashes" } }, "required": [ "path", "piece_length", "piece_hashes", "root_hash" ] }
- Fields:
- field leaf_hashes: list[Annotated[bytes, Len(min_length=32, max_length=32), PlainSerializer(func=_serialize_hash, return_type=PydanticUndefined, when_used=always)]] | None = None¶
SHA256 hashes of 16KiB leaf segments, if present.
- field path: Annotated[Path, AfterValidator(func=_is_rel)] [Required]¶
Path within torrent file
- Constraints:
func = <function _is_rel at 0x70dff2da6c00>
- field piece_hashes: list[Annotated[bytes, Len(min_length=32, max_length=32), PlainSerializer(func=_serialize_hash, return_type=PydanticUndefined, when_used=always)]] | None [Required]¶
hashes of each piece (the nth later of the merkle tree, determined by piece length).
When a file is smaller than a single piece, set explicitly to
None.
- classmethod from_path(path: Path, piece_length: Annotated[int, AfterValidator(func=_divisible_by_16kib), AfterValidator(func=_power_of_two)], path_root: Annotated[Path, AfterValidator(func=_is_abs)] | None = None, n_processes: int = 2, progress: bool = False, **kwargs: Any) MerkleTree[source]¶
Create a MerkleTree and return it with computed hashes
- Parameters:
path (Path) – Relative path to a file within a torrent directory. If absolute, must be beneath
path_rootpiece_length (V2PieceLength) – Piece length used for piece hashes
path_root (Path) – Absolute path that should serve as the root of the torrent, the path must be a relative or absolute path within it.
n_processes (int) – Number of processes to use while hashing, default is n_cpus
progress (bool) – Display progress while hashing
kwargs – Passed to
V2Hasher
- pydantic model MerkleTreeShape[source]¶
Helper class to calculate values when constructing a merkle tree, without needing to have a merkle tree itself.
Separated so that
MerkleTreecould just be a validated representation of the merkle tree rather than being the thing that hashes one, while also being able to validate the tree.Show JSON schema
{ "title": "MerkleTreeShape", "description": "Helper class to calculate values when constructing a merkle tree,\nwithout needing to have a merkle tree itself.\n\nSeparated so that :class:`.MerkleTree` could just be a\nvalidated representation of the merkle tree\nrather than being the thing that hashes one,\nwhile also being able to validate the tree.", "type": "object", "properties": { "file_size": { "title": "File Size", "type": "integer" }, "piece_length": { "title": "Piece Length", "type": "integer" } }, "required": [ "file_size", "piece_length" ] }
- Fields:
- field file_size: int [Required]¶
size of the file for which the merkle tree would be calculated, in bytes
- field piece_length: Annotated[int, AfterValidator(func=_divisible_by_16kib), AfterValidator(func=_power_of_two)] [Required]¶
piece length of the merkle tree
- Constraints:
func = <function _power_of_two at 0x70dff2da7600>
- validate_leaf_count(n_leaf_hashes: int) None[source]¶
Ensure that we have the right number of leaves for a merkle tree
- property n_pad_blocks: int[source]¶
Number of blank blocks required for padding when hashing.
Not strictly equivalent to the remainder to the nearest piece size, because we skip hashing all the zero blocks when we don’t need to. (e.g. when to balance the tree we need to compute a ton of empty piece hashes)
- pydantic model FileTree[source]¶
A v2 torrent file tree is like
folder/file1.png
file2.png
{ "folder": { "file1.png": { "": { "length": 123, "pieces root": b"<hash>", } } }, "file2.png": { "": { "length": 123, "pieces root": b"<hash>", } } }
Show JSON schema
{ "title": "FileTree", "description": "A v2 torrent file tree is like\n\n- `folder/file1.png`\n- `file2.png`\n\n.. code-block:: python\n\n {\n \"folder\": {\n \"file1.png\": {\n \"\": {\n \"length\": 123,\n \"pieces root\": b\"<hash>\",\n }\n }\n },\n \"file2.png\": {\n \"\": {\n \"length\": 123,\n \"pieces root\": b\"<hash>\",\n }\n }\n }", "type": "object", "properties": { "tree": { "$ref": "#/$defs/_FileTreeType" } }, "$defs": { "FileTreeItem": { "properties": { "length": { "title": "Length", "type": "integer" }, "pieces root": { "format": "binary", "maxLength": 32, "minLength": 32, "title": "Pieces Root", "type": "string" } }, "required": [ "length" ], "title": "FileTreeItem", "type": "object" }, "_FileTreeType": { "additionalProperties": { "anyOf": [ { "additionalProperties": { "$ref": "#/$defs/FileTreeItem" }, "propertyNames": { "const": "" }, "type": "object" }, { "$ref": "#/$defs/_FileTreeType" } ] }, "propertyNames": { "format": "binary" }, "type": "object" } }, "required": [ "tree" ] }
- Fields:
- field tree: Annotated[_FileTreeType, AfterValidator(func=_sort_keys)] [Required]¶
- Constraints:
func = <function _sort_keys at 0x70dff2db4220>
- classmethod flatten_tree(tree: Annotated[_FileTreeType, AfterValidator(func=_sort_keys)]) dict[str, FileTreeItem][source]¶
Flatten a file tree, mapping each path to the item description
- classmethod unflatten_tree(tree: dict[str, FileTreeItem]) Annotated[_FileTreeType, AfterValidator(func=_sort_keys)][source]¶
Turn a flattened file tree back into a nested file tree
- property flat: dict[str, FileTreeItem][source]¶
Flattened FileTree
- class PieceLayers(piece_length: int, piece_layers: dict[Annotated[bytes, Len(min_length=32, max_length=32), PlainSerializer(func=_serialize_hash, return_type=PydanticUndefined, when_used=always)], Annotated[list[Annotated[bytes, Len(min_length=32, max_length=32), PlainSerializer(func=_serialize_hash, return_type=PydanticUndefined, when_used=always)]], BeforeValidator(func=_validate_v2_hash, json_schema_input_type=PydanticUndefined), WrapSerializer(func=_serialize_v2_hash, return_type=PydanticUndefined, when_used=always)]], file_tree: FileTree)[source]¶
Constructor for piece layers, along with the file tree, from a list of files
Constructed together since file tree is basically a mapping of paths to root hashes - they are joint objects
- piece_layers: dict[Annotated[bytes, Len(min_length=32, max_length=32), PlainSerializer(func=_serialize_hash, return_type=PydanticUndefined, when_used=always)], Annotated[list[Annotated[bytes, Len(min_length=32, max_length=32), PlainSerializer(func=_serialize_hash, return_type=PydanticUndefined, when_used=always)]], BeforeValidator(func=_validate_v2_hash, json_schema_input_type=PydanticUndefined), WrapSerializer(func=_serialize_v2_hash, return_type=PydanticUndefined, when_used=always)]]¶
mapping from root hash to concatenated piece hashes
- Type:
piece layers
- classmethod from_trees(trees: list[MerkleTree] | MerkleTree, base_path: Path) PieceLayers[source]¶
- classmethod from_paths(paths: list[Annotated[Path, AfterValidator(func=_is_rel)]], piece_length: int, path_root: Path, n_processes: int = 2, progress: bool = False, **kwargs: Any) PieceLayers[source]¶
Hash all the paths, construct the piece layers and file tree
- Parameters:
paths (list[Path]) – List of relative paths within some path root
piece_length (V2PieceLength) – piece length valid for v2 torrents
path_root (Path) – Root directory that contains
pathsn_processes (int) – number of processes to use for parallel processing. Default is n_cpus
progress (bool) – Display progress while hashing
kwargs – passed to
V2Hasher
- pydantic model V2PieceRange[source]¶
A byte range that corresponds to a file or a piece within a v2 file.
If the length of the range is smaller than the piece length: if range_start is 0 we assume this range represents a whole file, otherwise we assume that the piece is the last piece in the file.
If the range represents a whole file, the piece_hash should be None and only the root hash should be given.
Show JSON schema
{ "title": "V2PieceRange", "description": "A byte range that corresponds to a file or a piece within a v2 file.\n\nIf the length of the range is smaller than the piece length:\nif range_start is 0 we assume this range represents a whole file,\notherwise we assume that the piece is the last piece in the file.\n\nIf the range represents a whole file, the piece_hash should be None\nand only the root hash should be given.", "type": "object", "properties": { "piece_idx": { "title": "Piece Idx", "type": "integer" }, "path": { "title": "Path", "type": "string" }, "range_start": { "title": "Range Start", "type": "integer" }, "range_end": { "title": "Range End", "type": "integer" }, "piece_length": { "title": "Piece Length", "type": "integer" }, "file_size": { "title": "File Size", "type": "integer" }, "piece_hash": { "anyOf": [ { "format": "binary", "maxLength": 32, "minLength": 32, "type": "string" }, { "type": "null" } ], "default": null, "title": "Piece Hash" }, "root_hash": { "format": "binary", "maxLength": 32, "minLength": 32, "title": "Root Hash", "type": "string" }, "full_path": { "title": "Full Path", "type": "string" } }, "required": [ "piece_idx", "path", "range_start", "range_end", "piece_length", "file_size", "root_hash", "full_path" ] }
- Fields:
- field full_path: str [Required]¶
Path to be used with webseeds, includes info.name in the case of multifile torrents, so the webseed base can be directly joined with full_path
- field piece_hash: Annotated[bytes, Len(min_length=32, max_length=32), PlainSerializer(func=_serialize_hash, return_type=PydanticUndefined, when_used=always)] | None = None¶
- field piece_length: Annotated[int, AfterValidator(func=_divisible_by_16kib), AfterValidator(func=_power_of_two)] [Required]¶
- Constraints:
func = <function _power_of_two at 0x70dff2da7600>
- field root_hash: Annotated[bytes, Len(min_length=32, max_length=32), PlainSerializer(func=_serialize_hash, return_type=PydanticUndefined, when_used=always)] [Required]¶
- Constraints:
min_length = 32
max_length = 32
func = <function _serialize_hash at 0x70dff2d99b20>
return_type = PydanticUndefined
when_used = always
- validate_data(data: list[bytes]) bool[source]¶
Validate 16KiB chunks of data against the provided piece or root hashes.
If the indicated range is smaller than the piece length, padding is added to the end to balance the merkle tree. Unlike with v1, the user does not need to add zero-padding to the data since it is unambigious from the piece range description.
- property tree_shape: MerkleTreeShape¶