btclib.p2p package¶
Submodules¶
btclib.p2p.address module¶
Where a peer is, what it offers, and the addr message that gossips it.
Bitcoin Core’s CAddress, of src/protocol.h, in the version-1 encoding its SERIALIZE_METHODS writes with Encoding::V1: eight octets of service flags, then the sixteen of address and two of port that CService writes under it. BIP155’s addrv2 is a different encoding of the same idea and is btclib.p2p.addrv2, not this module.
Two classes for what Core’s prose calls one structure, because the octets are not the same: a version message’s two addresses carry no timestamp and an addr message’s carry four octets of one in front. Core writes that as a serialization parameter – CAddress::SerParams with Format::Disk, Format::Network – and its test framework as CAddress.deserialize(f, with_time=True). A parameter is what btclib_node has too, and it is where that implementation loses a round trip: a NetworkAddress parsed with version_msg=True is forced to time=0 rather than left without one, so the field says zero where it means absent, and an addr entry cannot be told from a version one by looking at it. Here Version.addr_recv is annotated NetworkAddress and Addr.addresses holds TimestampedNetworkAddress, so putting one where the other belongs is a call mypy refuses rather than octets a peer refuses.
The port is big-endian and it is the only field of this protocol that is. Core writes it through Using<BigEndianFormatter<2>>(obj.port), in CService’s SERIALIZE_METHODS of src/netaddress.h, network byte order being what a port has been since sockets: everything else in a p2p message, this structure’s service flags and timestamp included, is little-endian. A self-consistent round trip cannot see this being wrong, which is why the test module for this one is driven by a captured addr message whose port reads 8333 one way and 36128 the other.
The address is sixteen octets, always, with IPv4 mapped into them. ::ffff:a.b.c.d – ten NUL octets, two 0xff, then the four of the v4 address – is what a v4 peer is on the wire, and the sixteen octets are what is held here rather than a narrower form plus a tag for which it is. That is the second thing btclib_node departs from and the second round trip it loses: it stores four octets for a v4 address and sixteen for a v6 one, so an IPv6 address that happens to begin with the mapping prefix parses back as the IPv4 address it is not, and its own test works around the collision with an unexplained 49. ipaddress.IPv6Address is .packed in one direction and the constructor in the other, exactly and for every value, and .ipv4_mapped is what answers the question the tag was for.
- class btclib.p2p.address.Addr(addresses: Sequence[TimestampedNetworkAddress] = (), *, check_validity: bool = True)[source]¶
Bases:
PayloadThe addr message: peers this node knows of, with when it saw them.
A count and that many TimestampedNetworkAddress, which is what Core’s msg_addr writes and what its ProcessMessage reads back.
The count is bounded before anything is built, at Core’s MAX_ADDR_TO_SEND: vAddr.size() > MAX_ADDR_TO_SEND is a Misbehaving there, so a message above it is one no peer accepts. The bound is checked in parse off the count and before the loop, where it is the whole point of having one – the count is the peer’s to choose, and btclib’s own var_int.parse allows 33,554,432 of them, which is the number of thirty-octet objects an implementation without this check builds out of nine octets. That check does not answer to check_validity, on Message.parse’s reasoning: a defence a caller can turn off is not one.
A tuple and not a list, so that a frozen Addr is what its fields say it is: dataclasses.replace is what changes the addresses in one, as it is for every other frozen class here.
- class btclib.p2p.address.NetworkAddress(services: int = <ServiceFlags.NODE_NONE: 0>, ip: IPv4Address | IPv6Address | str | bytes = '::', port: int = 0, *, check_validity: bool = True)[source]¶
Bases:
objectWhere a peer is and what it offers: (services, ip, port).
Bitcoin Core’s CService with the service flags in front of it, in the twenty-six octets a CAddress writes under Encoding::V1 with no timestamp in front – which is what a version message’s addr_recv and addr_from are. TimestampedNetworkAddress is the thirty-octet form an addr message carries, and the module docstring is why they are two classes.
ip is an ipaddress.IPv6Address and is always sixteen octets, an IPv4 peer being ::ffff:a.b.c.d: ip.ipv4_mapped is the v4 address where there is one and None where there is not, which is the question a caller would otherwise keep a tag for. The constructor takes any of the spellings IPAddress names, so “10.0.0.1” and “::ffff:10.0.0.1” build the one object – as they must, being the one peer.
services is a ServiceFlags, which is an int carrying the bits it cannot name; port is the one big-endian field in this protocol.
Frozen and hashable, all three fields being immutable: an address is a value, it is what an address database keys on, and dataclasses.replace is what moves one to another port.
No to_dict and no from_dict, for the reason Message has none: those agree with a json shape somebody else writes, and the shape Core’s rpc renders a peer as – getpeerinfo’s “addr” – is a formatted string and this structure’s fields spread across a dozen other keys, so the pair would be inventing one rather than reading one.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) NetworkAddress[source]¶
Return the address the twenty-six octets describe.
- class btclib.p2p.address.ServiceFlags(*values)[source]¶
Bases:
IntFlagThe services a node advertises, Bitcoin Core’s ServiceFlags.
src/protocol.h, spelled as Core spells it, and a bit set rather than a table: the eight octets are a bitfield, an unknown bit is a service this library has not heard of rather than an error, and Core says so where it reserves bits 24-31 “for temporary experiments” and sends everything else through the BIP process.
An IntFlag is what round-trips such a bit: a value with bits no member names keeps them and compares equal to the integer it was built from, so ServiceFlags(1 << 40) serializes back to the octets it was parsed from. The members are what a caller reads the named bits with – ServiceFlags.NODE_WITNESS in flags – and neither parse nor assert_valid consults them.
Bit 1 is absent because Core removed it: it was BIP64’s NODE_GETUTXO, and a version still carrying it is exactly the unnamed bit above. serviceFlagsToStr is Core’s own answer to the same question, and it answers “UNKNOWN[…]” rather than refusing.
- class btclib.p2p.address.TimestampedNetworkAddress(timestamp: int = 0, address: NetworkAddress | None = None, *, check_validity: bool = True)[source]¶
Bases:
objectOne entry of an addr message: when a peer was last seen, and where.
The thirty octets Bitcoin Core writes for a CAddress on the network – four of nTime, then the twenty-six a NetworkAddress is. The timestamp is unsigned and four octets wide where a version message’s is signed and eight, which is the second reason these are two structures rather than one with a flag: the same name in Core’s prose is not the same field.
Composed rather than inherited, so that a Version cannot be handed one: a subclass of NetworkAddress would satisfy that annotation and serialize four octets nobody asked for, which is the trap this module’s docstring names in the implementation that has it.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) TimestampedNetworkAddress[source]¶
Return the entry the thirty octets describe.
btclib.p2p.addrv2 module¶
BIP155’s addrv2, the sendaddrv2 that asks for it, and the id table.
The message that gossips an address of any network, and the one a peer sends to say it understands the first. BIP155 is the specification and is unusually precise; Core’s CAddress under V2_NETWORK, of src/protocol.h and src/netaddress.h, is the implementation, and where the two read differently this module says so and follows the BIP.
An entry is addr’s idea in a different encoding, not a variant of it: four octets of timestamp, then the service flags as a CompactSize, a one-octet network id, the address as var_bytes, and the big-endian port. Nothing about it is an addr entry’s octets, and the address field is not an IP address at all – a TORV3 one is an ed25519 public key.
A class of its own, and one rather than two. NetworkAddress holds an ipaddress.IPv6Address, which a TORV2, TORV3 or I2P address is not, so reuse is refused by the field before it is refused by anything else: giving that field a union type would put “which of these is it” on every existing caller of .ip, which is the cost issue #1098 declined when it made TimestampedNetworkAddress a second class rather than a flag on the first. A caller therefore has three types where the protocol has three encodings – a version’s address, an addr’s, and this one – and the type is what says which message an address came off.
One and not two because BIP155 defines one record: time is in its table unconditionally, no message carries the untimestamped form, and the split that gave addr two classes has nothing here to divide. NetworkAddressV2 is therefore the counterpart of TimestampedNetworkAddress rather than of NetworkAddress, and “V2” is Core’s word for the encoding – CAddress::V2_NETWORK, SerializeV2Stream – rather than a second version of the class above it.
An unknown network id round-trips. It is the property the format exists for: a new network is meant to need no new message, so a parser that refused an id it had not heard of could not read the message the next BIP makes ordinary. _bip155_network_from_int hands back the plain int where no member names the id, which is what InventoryType does for a type code and the envelope does for a command. Core drops such an address instead – SetNetFromBIP155Network returns false and UnserializeV2Stream consumes the octets into a default CNetAddr – and it is right to, being a node: BIP155 says clients SHOULD NOT gossip addresses of networks they cannot validate. That is a rule about what to relay, and this package relays nothing; keeping the octets is what lets the caller apply it.
A known id whose address is the wrong length is refused, and the message with it. BIP155: “Clients SHOULD reject messages that contain addresses that have a different length than specified in this table for a specific network ID, as these are meaningless.” Core implements exactly that, SetNetFromBIP155Network throwing std::ios_base::failure for every id it knows, which fails the whole message rather than the entry. So does assert_valid here, and refusing is what a codec can mean by it: dropping the entry would leave a message that serializes back one address shorter than it arrived, and there is no value of a field that means “this one was ignored”.
What BIP155 does say to ignore is a different thing, and neither half of it is refused here: a TORV2 address, and an IPV6 address inside a range reserved for embedding another network – ::ffff:0:0/96 and OnionCat’s fd87:d87e:eb43::/48. Both are receive policy, about whether an address is worth keeping rather than whether the octets decode, and Core carries them out by parsing the entry and marking the result invalid. A parser that refused them would refuse a message Core accepts. is_embedded_ipv6 is the second rule as a public predicate, so a caller applying it does not have to learn the two prefixes for itself.
`TORV2` and `YGGDRASIL` are in the table, where Core acts on neither. BIP155 reserves both, and says “Further network ID numbers MUST be reserved in a new BIP document”, so 3 and 7 mean what they mean for good. Core names TORV2 in its own BIP155Network and nowhere else – SetNetFromBIP155Network has no case for it, so an address under it falls through to the unknown-id path and is dropped, and the test framework’s ADDRV2_NET_NAME has no entry for it. YGGDRASIL Core has not got at all, Yggdrasil being carried there as ordinary IPv6, so an id 7 from a real Yggdrasil peer is an unknown network to a Core node today.
Naming them is not offering them: an id round-trips whether or not a member names it, so what the member changes is only whether a caller reading a captured message sees BIP155Network.TORV2 or 3 – and a table with a hole in it is one somebody eventually reuses. The rule the tree already follows is InventoryType’s: name what the specification names, and not what it merely reserves for the future.
The port is big-endian, as it is in addr, and the evidence is not a round trip: tests/p2p/addrv2_test.py is driven by the addrv2 payload of Core’s own netbase_tests.cpp, whose last entry reads port f1f2 – 61938 one way and 62193 the other.
The service flags are a `CompactSize` with no range check on it, which is the one place btclib’s var_int default is wrong for this format. var_int.parse caps at Core’s MAX_SIZE, 33,554,432, because every var_int btclib otherwise reads is a length or a count; these octets are a 64-bit bitfield, and Core reads them through Using<CompactSizeFormatter<false>>, the false being RangeCheck. Bit 25 is 33,554,432 exactly – inside Core’s “reserved for temporary experiments” range, bits 24 to 31 – so the default cap starts refusing peers one bit above it.
The address itself is opaque octets, and no rendering is offered. That is what BIP155 bought: the field’s meaning is the network id’s, so a type that decoded it would have to hold every network’s format and would be wrong about the next one. IPv4Address(entry.address) and its v6 twin are a caller’s one line wherever the octets are an IP address at all; a .onion name is SHA3-256 and a checksum over them, and a .b32.i2p name is base32 of them, neither of which is a codec’s work.
`network_address`, `addr_entry` and `peer_from_addr_entry` are the translation to and from `btclib.p2p.address`’s two classes, and `can_addrv1` is the question a caller asks first. btclib-node holds a peer as a NetworkAddressV2 – BIP155’s record being the only encoding wide enough for every network a peer can be on – and still speaks addr to a peer that has not sent sendaddrv2, so it wrote the translation. Each is a function of the two types this package already owns and of the BIP that relates them, which is why it belongs here rather than beside a socket: network_address refuses a network addr cannot carry, on can_addrv1’s own question, and addr_entry and peer_from_addr_entry are what stand on either side of that refusal, adding and reading the timestamp TimestampedNetworkAddress carries and NetworkAddressV2 already has.
- class btclib.p2p.addrv2.AddrV2(addresses: Sequence[NetworkAddressV2] = (), *, check_validity: bool = True)[source]¶
Bases:
PayloadThe addrv2 message: peers of any network, with when they were seen.
A count and that many NetworkAddressV2, which is Core’s msg_addrv2 and what ProcessMessage reads through the same branch it reads an addr with, CAddress::V2_NETWORK in place of V1_NETWORK being the whole of the difference there.
No base shared with Addr, where Inv and GetData have one: those two are one body under two commands, and these two are two bodies – a TimestampedNetworkAddress and a NetworkAddressV2 are different octets, so what a base could hold is the word “count”.
The count is bounded before anything is built, at MAX_ADDR_TO_SEND, which is BIP155’s thousand and Core’s constant for both commands. As in Addr.parse the check is off the count and before the loop, and it does not answer to check_validity.
A tuple and not a list, so that a frozen AddrV2 is what its field says it is; dataclasses.replace is what changes the addresses in one.
- class btclib.p2p.addrv2.BIP155Network(*values)[source]¶
Bases:
IntEnumThe network an addrv2 address belongs to, BIP155’s id table.
Every id of that table, spelled as its “Enumeration” column spells them. Core’s own name for the enum, of src/netaddress.h, taken rather than a NetworkId: “network” in btclib already means the chain – btclib.network.Network, and btclib.p2p.magic_from_network beside it – and these are IPv4, Tor and I2P.
TORV2 and YGGDRASIL are members where Core acts on neither, and the module docstring is why. An id no member names is not an error either, which is _bip155_network_from_int.
An IntEnum and not an IntFlag, for the reason InventoryType is one: these are exclusive kinds and not bits, so composing two of them would answer for a network that does not exist.
- class btclib.p2p.addrv2.NetworkAddressV2(timestamp: int = 0, services: int = <ServiceFlags.NODE_NONE: 0>, network_id: int = BIP155Network.IPV4, address: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00', port: int = 0, *, check_validity: bool = True)[source]¶
Bases:
objectOne entry of an addrv2: a peer, and the network it is on.
Bitcoin Core’s CAddress written with CAddress::V2_NETWORK, which is BIP155’s table of fields in its order: timestamp in four octets little-endian, services as a CompactSize, network_id in one octet, address as var_bytes, and port in two octets big-endian.
address is the octets and nothing more – the network id says how to read them, and the module docstring is why nothing here does. network_id is a BIP155Network where a member names the id and the plain int where none does, which is how an address of a network this library has not heard of comes back as it arrived. services is a ServiceFlags, which is an int carrying the bits it cannot name.
A port of zero is what BIP155 requires where a port means nothing for the network, and is a value like any other here.
Frozen and hashable, every field being immutable: an address is a value, it is what an address database keys on, and dataclasses.replace is what moves one to another port.
- assert_valid() None[source]¶
Refuse a field no width holds, and an address of the wrong length.
The length is BIP155’s table read against network_id: an id a member names fixes it, and a mismatch is what the BIP calls meaningless and what Core’s SetNetFromBIP155Network throws on. An id no member names fixes nothing, so MAX_ADDRV2_SIZE is the whole of what such an address is held to.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) NetworkAddressV2[source]¶
Return the entry the octets describe, the address bounded first.
MAX_ADDRV2_SIZE is checked off the length field and before the read it would size, which is where a bound is worth having: the length is the peer’s to choose. It does not answer to check_validity, on Message.parse’s reasoning – a defence a caller can turn off is not one – where the table check in assert_valid does, being a statement about the address rather than about what reading it costs.
- class btclib.p2p.addrv2.SendAddrV2(*, check_validity: bool = True)[source]¶
Bases:
PayloadThe sendaddrv2 message: no fields, and an empty payload.
Core’s msg_sendaddrv2, and the whole of that command: a peer that sends one is saying it understands addrv2 and would rather have it than addr.
BIP155 puts it in the handshake – it “MUST only be sent in response to the version message from a peer and prior to sending the verack message”, and Core disconnects a peer that sends one after the verack. That is a rule about when, which needs a connection to hold; this package has none, so the rule is documented here and nothing enforces it. The message lives beside addrv2 rather than in btclib.p2p.handshake because it is about nothing else.
parse refuses an octet, as Verack.parse does and for that class’s reason: this library refuses what follows an object everywhere else, and a sendaddrv2 with a payload is a message that serializes back without it. The two classes repeat three lines rather than share a base – keepalive._NoncePayload is a base because two commands have one body, and the absence of a body is not one to share.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) SendAddrV2[source]¶
Return a SendAddrV2, refusing any octet at all.
Octets are refused by assert_no_trailing, which is where the rule already is; a caller’s stream is left exactly where it was, a sendaddrv2 consuming nothing from one.
- btclib.p2p.addrv2.addr_entry(address: NetworkAddressV2) TimestampedNetworkAddress[source]¶
Return the addr entry a BIP155 record can_addrv1 allows.
- btclib.p2p.addrv2.can_addrv1(address: NetworkAddressV2) bool[source]¶
Answer whether an addr message has room for this peer.
Both IP networks and neither of the others: the question is about the network being an IP one at all, not about whether the address is otherwise worth dialling, which is a node’s own policy and not this package’s.
- btclib.p2p.addrv2.is_embedded_ipv6(address: NetworkAddressV2) bool[source]¶
Answer whether an IPV6 record’s octets are really another network’s.
BIP155’s two ignore rules together: a v4-mapped address, and one inside OnionCat’s range, once used to carry a TORv2 address the same way. Both are receive policy, about whether an address is worth keeping rather than whether the octets decode – the module docstring is why assert_valid does not apply this – so it is a caller deciding what to keep that does.
- btclib.p2p.addrv2.network_address(address: NetworkAddressV2) NetworkAddress[source]¶
Return the untimestamped form of a BIP155 record can_addrv1 allows.
What a version message’s two addresses are, and what an addr entry is built on. can_addrv1 is the question a caller asks first; the refusal here is what makes the answer binding rather than advisory, because the length would not catch it on its own: BIP155 gives cjdns and yggdrasil the sixteen octets an IPv6 address has, so IPv6Address would take either for an IP address and hand back a peer that is not the one that was gossiped.
- btclib.p2p.addrv2.peer_from_addr_entry(entry: TimestampedNetworkAddress) NetworkAddressV2[source]¶
Return the BIP155 record an addr entry describes.
An addr entry holds every address in sixteen octets, a v4 one mapped into them, where BIP155 gives the two networks different ids and different lengths: ip.ipv4_mapped is what tells them apart, and it is why this is not a field rename.
btclib.p2p.block_filters module¶
BIP157’s six messages: the compact filters, and the chain over them.
The client’s three requests and the server’s three answers. getcfilters asks for the filters of a height range and is answered by one cfilter per block; getcfheaders asks for the filter hashes of a range and is answered by one cfheaders holding them all; getcfcheckpt asks for every thousandth filter header and is answered by a cfcheckpt. BIP157 is the specification, and its field tables are the wire form; the layout Core writes is test/functional/test_framework/messages.py’s msg_getcfilters, msg_cfilter, msg_getcfheaders, msg_cfheaders, msg_getcfcheckpt and msg_cfcheckpt, and the handlers reading them are ProcessGetCFilters, ProcessGetCFHeaders and ProcessGetCFCheckPt of src/net_processing.cpp.
A `cfilter` holds the filter as octets, not as a `BasicBlockFilter`. The type code decides the format – BIP157: “Each type is identified by a one byte code, and specifies the contents and serialization format of the filter” – and only BASIC is defined, so octets under any other code are not a BIP158 filter and reading them as one would be inventing an answer. basic_filter is where a caller says the code is BASIC and gets the typed object, refused there if the Golomb stream does not decode.
That property costs nothing here because of the field order: BIP157 puts BlockHash before FilterBytes, and BasicBlockFilter.parse needs exactly that hash – it is the SipHash key the filter is built under, and the reason the hash is parse’s argument rather than something read back out of the octets. So the message carries everything the typed object needs, basic_filter takes no argument, and nothing about the seam is left to a caller to supply.
What a `cfilter` cannot be checked for is that its filter is its block’s, and saying so is the point rather than an omission. A block hash is a key and not a commitment: every thirty-two octets key some filter, so a filter and a hash that do not belong together decode exactly as a pair that does. What settles it is the block – from_block recomputes the filter – or the header chain a cfheaders carries, and both are the caller’s, this package holding no chain and no blocks it did not receive.
An unrecognized filter type round-trips as a plain integer, which is what InventoryType and BIP155’s network id do here, and BIP157’s own text is what decides it rather than that precedent: “Nodes receiving getcfilters with an unsupported filter type SHOULD NOT respond” is a rule about answering, and a rule about answering can only be applied by something that has read the message. Core reads the three requests that way, ProcessGetCFilters casting the octet to BlockFilterType and PrepareBlockFilterRequest disconnecting the peer afterwards.
Core’s BlockFilter, which serializes the very octets a cfilter carries, does refuse an unknown type instead – Unserialize throws std::ios_base::failure(“unknown filter_type”) – and the reason is the line after it: it builds the Golomb parameters on the spot, and BuildParams has none for a type it does not know. That is the work this module defers to basic_filter, which is why the same octets can be held here and cannot be held there; Core needs no receiving path for a cfilter at all, being the server of these six messages rather than the client.
BlockFilterType therefore names BASIC and nothing else. Core’s own enum has a second member, INVALID = 255, and it is not here: it is what BlockFilter::m_filter_type is initialized to and the case BuildParams returns false for – a filter with no type rather than a type a peer sends – where BIP158 defines the one code the protocol has. The same reason InventoryType does not name the composite BIP144 reserved and Core carries commented out.
`cfheaders` stores the filter hashes it carries, and derives the headers. The message holds a previous filter header and a vector of hashes; the headers are what a client computes from the two, and BIP157 sends the hashes precisely so that it has to. Storing the derived headers instead would be the question issue #1101 answered for headers’ always- zero transaction count, and the answer is the same: keep what the wire holds, so that every payload serializes back to the octets it came from. filter_headers is the derivation, as Inventory.is_witness is the reading of a bit rather than a second field – and it is the same block_filter.filter_header that BasicBlockFilter.header is, the general form being over a hash and the method the case where the filter is at hand.
cfcheckpt needs no such derivation: what it carries is filter headers, one per thousand blocks, and heights is the arithmetic BIP157 states over them.
Every hash is held in the order a block explorer prints it and reversed on the wire, as everywhere else in this package: BlockHeader.hash is what a stop_hash is compared against, BasicBlockFilter.hash is what a cfheaders entry is, and BasicBlockFilter.header is what a cfcheckpt entry is. A caller must not have to reverse one of them.
Two of BIP157’s three bounds are on a range this package cannot resolve, and the third is a count, which is the one a parser can hold. MAX_GETCFILTERS_SIZE and MAX_GETCFHEADERS_SIZE bound the distance from StartHeight to the height of StopHash, and a hash is a height only to something holding the chain, so getcfilters and getcfheaders carry the constants in their docstrings for the caller that has one and check neither. cfheaders’ FilterHashesLength is BIP157’s “MUST NOT be greater than 2,000” and is checked before the loop that allocates on it, as every count in this package is.
cfcheckpt’s vector is bounded by no constant, and none is invented for it: BIP157 bounds it by the length of the chain, Core sizes it as stop_index->nHeight / CFCHECKPT_INTERVAL when it writes one and reads none, and what stands in front of the loop here is the octets – read_exactly refuses the first entry past the end, so the vector cannot outgrow the payload, and the payload is the envelope’s MAX_PROTOCOL_MESSAGE_LENGTH. A second constant would be a number to be wrong about in a third place, which is btclib.p2p.data’s argument for bounding no message length.
- class btclib.p2p.block_filters.BlockFilterType(*values)[source]¶
Bases:
IntEnumWhat a filter type code names, Core’s BlockFilterType.
src/blockfilter.h, and the one code BIP158 defines: “The initial filter types are defined separately in BIP 158”, which defines BASIC and nothing after it.
Core’s enum has a second member, INVALID = 255, and it is not here. That value is what BlockFilter::m_filter_type is initialized to and the one case BuildParams answers false for – a filter that has no type, rather than a type a peer may send – and a library naming it would be publishing a code the protocol has not got. InventoryType leaves out the composite BIP144 reserved for the same reason.
An IntEnum, and _block_filter_type_from_int is what keeps a code no member names from being an error: the module docstring has why an unsupported type is a rule about answering rather than about reading.
- class btclib.p2p.block_filters.CFCheckpt(filter_type: int = BlockFilterType.BASIC, stop_hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', filter_headers: Sequence[bytes | str | bytearray | memoryview] = (), *, check_validity: bool = True)[source]¶
Bases:
PayloadThe cfcheckpt message: a filter header every thousand blocks.
Bitcoin Core’s msg_cfcheckpt: the type code, the stop hash, and the vector of filter headers. These are headers and not hashes, so nothing is derived from them – the contrast with cfheaders one class up, which sends the hashes so that the client does the chaining. filter_headers is the field here and the derivation there, which is the one name a caller wants off either message.
heights is what BIP157 says the entries are of: “one entry for each block on the chain terminating in StopHash, where the block height is a multiple of 1,000 greater than 0”.
No count bound, where every other vector in this package has one: BIP157 bounds this one by the length of the chain and Core reads no cfcheckpt at all, so there is no constant to hold it to and none is invented. The module docstring has what stands in front of the loop instead.
Frozen and hashable; dataclasses.replace is what changes the vector.
- property heights: list[int]¶
Return the block height each filter header is that of.
BIP157’s rule read off the vector, CFCHECKPT_INTERVAL being Core’s name for the thousand: the entries are in ascending order by height, so the first is the header of block 1,000 and the last is the highest multiple of a thousand at or below the height of the stop hash.
- class btclib.p2p.block_filters.CFHeaders(filter_type: int = BlockFilterType.BASIC, stop_hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', previous_filter_header: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', filter_hashes: Sequence[bytes | str | bytearray | memoryview] = (), *, check_validity: bool = True)[source]¶
Bases:
PayloadThe cfheaders message: the filter hashes a header chain is built of.
Bitcoin Core’s msg_cfheaders: the type code, the stop hash, the filter header before the first block of the range, and the vector of filter hashes. Every hash is in display order, BasicBlockFilter.hash’s and BasicBlockFilter.header’s.
The hashes are the field and the headers are derived, which is what BIP157 sends: a client that was handed the headers would have nothing left to check, where the hashes plus one previous header chain into headers it computed itself. filter_headers is that derivation; the module docstring is where storing it instead is refused.
previous_filter_header is thirty-two zero octets for a range starting at the genesis block, which is BIP157’s definition of the header before the first one.
Frozen and hashable, every field being immutable; dataclasses.replace is what changes the vector.
- property filter_headers: tuple[bytes, ...]¶
Return the filter header of each block of the range, in order.
BIP157: a filter header is “the double-SHA256 of the concatenation of the filter hash with the previous filter header”, so the vector plus previous_filter_header is a chain, and the last entry is the header a client compares against what another peer told it. block_filter.filter_header is the one step, the same one BasicBlockFilter.header takes where the filter itself is at hand.
Every hash is in display order, so every header answered is too. A tuple, as CFCheckpt.filter_headers is: one name over the two messages, and one type with it.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) CFHeaders[source]¶
Return the hashes the payload carries, the count bounded first.
BIP157’s “FilterHashesLength MUST NOT be greater than 2,000”, checked before the loop that allocates on it: the count is the peer’s to choose and btclib’s var_int.parse allows 33,554,432 of anything.
- class btclib.p2p.block_filters.CFilter(filter_type: int = BlockFilterType.BASIC, block_hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', filter_bytes: bytes | str | bytearray | memoryview = b'', *, check_validity: bool = True)[source]¶
Bases:
PayloadThe cfilter message: one block’s filter, and the block it is of.
Bitcoin Core’s msg_cfilter: the type code, the block hash, and the serialized filter behind a CompactSize length. block_hash is in display order, BlockHeader.hash’s.
filter_bytes is BIP157’s FilterBytes – what BasicBlockFilter.serialize writes, the element count and the Golomb-Rice set – and is held as octets: what they encode is the type code’s to say, and only BASIC says anything. basic_filter is the typed reading, and the module docstring is where that is argued and where what this message cannot be checked for is written down.
Frozen and hashable, both octet fields being immutable, which is what holding the filter as octets rather than as a mutable BasicBlockFilter buys.
- assert_valid() None[source]¶
Refuse a type or a block hash the fields cannot hold.
The filter octets are not decoded, whatever the type code says: basic_filter is where they are read as BIP158’s, and refusing them here would make the same octets parse under a type code nobody has defined and fail under the one that is. They round-trip either way, which is the property this package keeps.
Core’s BlockFilter::Unserialize does the opposite and throws “unknown filter_type”, because it builds the Golomb parameters as it reads; deferring that to basic_filter is the whole of the difference, and the module docstring is where it is argued.
Nor are they asked anything else. bytes_from_octets is what __init__ coerced them with and there is no width they must have, so unlike the two hash fields there is nothing left here to refuse – a cfilter of an empty filter is a message BIP158’s own vector file holds.
- property basic_filter: BasicBlockFilter¶
Return the filter these octets are, keyed on this block hash.
The reading and not the field: a cfilter of any other type carries octets no BIP defines, so this is where a caller says the type is BASIC and is refused if it is not. What BasicBlockFilter.parse then refuses is a Golomb stream that does not decode – an element count the bits fall short of, a delta past the range, an octet the deltas never reached.
No argument, which is the seam this message closes: BlockHash precedes FilterBytes in BIP157’s table, so the hash the filter is keyed by arrived with it.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) CFilter[source]¶
Return the filter octets the payload carries, and their block.
NumFilterBytes gets no bound of its own: var_bytes.parse reads the length and then reads from the stream, so what it can build is what the payload holds, and what the payload holds is the envelope’s MAX_PROTOCOL_MESSAGE_LENGTH. A filter has no other limit to be held to – BIP158 bounds the element count and BasicBlockFilter.parse checks that, over octets a caller has already been handed.
- class btclib.p2p.block_filters.GetCFCheckpt(filter_type: int = BlockFilterType.BASIC, stop_hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', *, check_validity: bool = True)[source]¶
Bases:
PayloadThe getcfcheckpt message: the checkpoints up to a block.
Bitcoin Core’s msg_getcfcheckpt: a type code and a stop hash, and the one request of the three with no start height – a checkpoint chain always begins at the genesis block, so what a client asks for is only where it ends.
A class of its own rather than a third _FilterRangeRequest: two fields are not three, and a start_height here would be a field no message carries.
Frozen and hashable, both fields being immutable.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) GetCFCheckpt[source]¶
Return the chain end the payload asks the checkpoints of.
- class btclib.p2p.block_filters.GetCFHeaders(filter_type: int = BlockFilterType.BASIC, start_height: int = 0, stop_hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', *, check_validity: bool = True)[source]¶
Bases:
_FilterRangeRequestThe getcfheaders message: the filter hashes of a range of blocks.
Bitcoin Core’s msg_getcfheaders, and the same three fields as getcfilters for a range twice as long: BIP157 bounds this one at “strictly less than 2,000”, limits.MAX_GETCFHEADERS_SIZE, and it is unchecked here for the reason above. The answer is one cfheaders however long the range, the hashes being fixed width.
- class btclib.p2p.block_filters.GetCFilters(filter_type: int = BlockFilterType.BASIC, start_height: int = 0, stop_hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', *, check_validity: bool = True)[source]¶
Bases:
_FilterRangeRequestThe getcfilters message: the filters of a range of blocks.
Bitcoin Core’s msg_getcfilters. The answer is one cfilter per block, “sequentially in order by block height”, which is the one request here whose answer is many messages.
`limits.MAX_GETCFILTERS_SIZE` is not checked here: BIP157 bounds the range, “the difference MUST be strictly less than 1000”, and the far end of it is a hash. Turning that hash into a height needs the chain, which this package does not hold, so the bound belongs to the caller that does – and the constant is in btclib.p2p.limits under Core’s own name for it.
btclib.p2p.compact_blocks module¶
BIP152’s four messages, and the block a compact one is put back into.
sendcmpct negotiates, cmpctblock announces a block as a header and a vector of six-octet short ids, and getblocktxn/blocktxn are the round trip that fills what the receiver had not got. BIP152 is the specification and its field tables are the wire form; Core’s src/blockencodings.h and .cpp are the implementation, and its test/functional/test_framework/messages.py’s msg_sendcmpct, msg_cmpctblock, msg_getblocktxn and msg_blocktxn the layout.
This is the one module of the package that carries an algorithm, and the algorithm rather than the layout is where a compact block goes wrong: an encoder and a decoder that agree with each other and disagree with every peer is what a format built out of a keyed hash and a differential encoding invites, and no round-trip test can see it. So each of the three pieces below says where its answer comes from.
The short id key is derived per message, from the header and the nonce. BIP152: single-SHA256 of the block header serialization with the nonce appended little-endian, and “the first two little-endian 64-bit integers from the above hash” are the SipHash-2-4 key. Core’s CBlockHeaderAndShortTxIDs::FillShortTxIDSelector is those three lines, shorttxidhash.GetUint64(0) and GetUint64(1) being the two words. short_id_key is the pair, and btclib.hashes.siphash takes exactly it.
A short id is the low 48 bits of the siphash of a wtxid, not of a transaction id, and the octets hashed are the hash in its internal order: Core’s GetShortID hashes wtxid.ToUint256(), which is what Wtxid serializes, and this package holds every hash in the order a block explorer prints – so short_id reverses what it is handed, as everything else here reverses on the wire. Getting either of the two wrong produces a library that reconstructs its own blocks and no peer’s, which is why the vectors are a mainnet block already in the tree with its short ids computed a second way.
Every index is differentially encoded, in prefilledtxn and in getblocktxn alike: BIP152 writes “the difference between the current index and the previous index, minus one”, so a first index of 0 is index 0 and a second 0 after it is index 1. The fields below hold the absolute index, which is what BIP152’s own Purpose column says the field is – “The index into the block at which this transaction is” – and the difference is taken and undone in serialize and parse. The alternative, holding the wire’s own value, is what Core’s PrefilledTransaction::index does, and its comment is the reason not to: “Used as an offset since last prefilled tx in CBlockHeaderAndShortTxIDs, as a proper transaction-in-block-index in PartiallyDownloadedBlock” – one field, two meanings, told apart by which object is holding it. Nothing is lost by storing the absolute index: over indexes that strictly increase the two are the same sequence written twice, so the octets round-trip either way. btclib_node holds the wire’s value and never undoes the difference, which is a decoder that agrees with its own encoder and with nothing else (btclib_node issue #20).
Version 2 alone is implemented, and version 1 is not. The two differ in one field and one hash: version 2 writes the transactions inside cmpctblock and blocktxn with their witnesses, “using the same format as responses to getdata MSG_WITNESS_TX”, and computes short ids over the wtxid where version 1 computes them over the txid. Core’s CMPCTBLOCKS_VERSION is 2 and ProcessMessage answers a sendcmpct of any other version by ignoring it – if (sendcmpct_version != CMPCTBLOCKS_VERSION) return; – so version 1 is a dialect no current peer will speak, and offering it would be the library offering what no peer accepts.
What that costs is less than it reads, and saying so is the honest half of the decision. A version 1 message’s octets still round-trip here unchanged, because a version 1 sender writes no witness and Tx.serialize(include_witness=True) writes none either where there is none to write – the marker goes in only when a witness follows it, so the two encodings are the same octets for every transaction a version 1 peer sends. There is therefore no include_witness field here, where btclib.p2p.data has one on tx and block: there the two encodings are both live today, MSG_TX and MSG_WITNESS_TX being two things a peer may ask for, and here the stripped one belongs to a version Core no longer speaks. And short_id takes a hash rather than a transaction, so the version 1 derivation is the same method over Tx.id for a caller who has a use for it: the derivation is one, and which hash goes into it is the version’s to say.
Reconstruction is a module function and not a payload method, and what it answers with when the pool is short is a list of indexes rather than an exception. reconstruct takes a compact block and a pool of candidate transactions and returns a PartialBlock, whose missing_indexes is exactly what a getblocktxn is built from and whose fill takes the blocktxn that answers it. Putting it on CmpctBlock was the other shape and is refused: every method of a payload type in this package is about the payload’s own octets, and matching a mempool against a block is not serialization – it is the one thing here that takes something the message did not carry. Core splits it the same way and in the same two steps, PartiallyDownloadedBlock::InitData and ::FillBlock; the class here is not called partially downloaded, which is a word for a package that opens no socket.
Short ids collide, and the two collisions are two different answers. A cmpctblock whose own short ids are not unique cannot be reconstructed at all – there is no index to ask for, because two positions want one transaction – so reconstruct refuses it and BIP152’s answer is to re-request the block. Core’s is if (shorttxids.size() != cmpctblock.shorttxids.size()) return READ_STATUS_FAILED; // Short ID collision. That refusal is reconstruct’s and not assert_valid’s: a message whose short ids collide is a message a peer legitimately sends, BIP152 saying that nodes “MUST NOT be penalized for such collisions”, so it has to parse and serialize back. The other collision is two pool transactions answering one short id, and there the answer is that the index stays missing and is requested – Core drops both and says why, “eating a round-trip due to FillBlock failure would be annoying”. A reconstructor that takes the first match instead is wrong in a way its own tests cannot see.
The bound is one number under two roles, which is Core’s own spelling. limits.MAX_BLOCK_TX_INDEX bounds the count in front of every vector here and the value of every index, because Core writes std::numeric_limits<uint16_t>::max() at both: once in CBlockHeaderAndShortTxIDs’s deserializer, which throws “indexes overflowed 16 bits” on a BlockTxCount() past it, and once in DifferenceFormatter::Unser, which throws “differential value overflow” on an index past it. Core applies a second bound in InitData, MAX_BLOCK_WEIGHT / MIN_SERIALIZABLE_TRANSACTION_WEIGHT, and it is not published here because it is the weaker of the two and would be a dead constant: a hundred thousand is more than sixteen bits hold.
What no message here is refused for is worth naming, each being a rule about a peer rather than about octets. A getblocktxn with an empty index vector is a peer Core disconnects – “No legitimate reason to send indexes empty” – and is a well-formed message. A sendcmpct naming a version this library does not speak is the whole point of the field, and is read and written unchanged. And whether a blocktxn answers the getblocktxn that was sent is a question about a connection, which this package does not hold.
- class btclib.p2p.compact_blocks.BlockTxn(block_hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', transactions: Sequence[Tx] = (), *, check_validity: bool = True)[source]¶
Bases:
PayloadThe blocktxn message: the transactions a getblocktxn asked for.
BIP152’s BlockTransactions and Core’s class of that name: the block hash and the transactions, “exactly and only each transaction which is present in the appropriate block at the index specified in the getblocktxn indexes list, in the order requested”. block_hash is in display order; the transactions are written with their witnesses, which is BIP152 version 2 and what the module docstring argues.
That the transactions are the ones that were asked for is a property of a connection and not of these octets, so nothing here checks it: what does is PartialBlock.fill, which puts them in the positions the same reconstruction found missing and hands back a Block whose merkle root either commits to them or does not.
Frozen, and not hashable: Tx is a mutable dataclass.
- class btclib.p2p.compact_blocks.CmpctBlock(header: BlockHeader, nonce: int = 0, short_ids: Sequence[int] = (), prefilled_txns: Sequence[PrefilledTransaction] = (), *, check_validity: bool = True)[source]¶
Bases:
PayloadThe cmpctblock message: a header, short ids, and a few transactions.
BIP152’s HeaderAndShortIDs, which is the whole of the payload – so there is one class here and not a message wrapping a structure, as there is for getblocktxn and blocktxn too. Core’s CBlockHeaderAndShortTxIDs is the same five fields, the two vector lengths being what var_int writes rather than fields of their own.
short_ids are the six-octet integers of the transactions the sender expects the receiver to have, in block order with the prefilled positions taken out; prefilled_txns are the ones it sends whole. Together they are the block: tx_count is their sum, which is BIP152’s “block tx count” read off either vector.
short_id is the derivation the ids come from and short_id_key the key it runs under, both of them functions of the header and the nonce and therefore of this message alone. The module docstring is where the derivation is stated and where what a wrong one would cost is.
Frozen, and not hashable: a PrefilledTransaction holds a mutable Tx. dataclasses.replace is what changes a vector.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) CmpctBlock[source]¶
Return the block this announces, both counts bounded first.
Each vector’s length is read against MAX_BLOCK_TX_INDEX before the loop that allocates on it, btclib’s var_int.parse allowing 33,554,432 of anything; the sum is what assert_valid holds to the same bound afterwards, which is where Core checks it too.
- serialize(*, check_validity: bool = True) bytes[source]¶
Return the header, the nonce, then the two vectors.
- short_id(wtxid: bytes | str | bytearray | memoryview) int[source]¶
Return the six-octet short id this message would carry for a hash.
The hash is a wtxid in BIP152 version 2, Tx.hash, and it is taken in the order this package holds every hash in – the order a block explorer prints – and reversed here, Core hashing the uint256 its own way round. What comes back is the siphash with its two most significant octets dropped, BIP152’s step three.
A hash and not a transaction, which is what leaves version 1 reachable without being offered: the same derivation over Tx.id is the version 1 short id, and which of the two hashes goes in is the negotiated version’s to say rather than this method’s.
- property short_id_key: tuple[int, int]¶
Return the (k0, k1) the short ids of this message are keyed on.
BIP152: single-SHA256 of the header serialization with the nonce appended little-endian, and the first two little-endian 64-bit integers of it. Core’s FillShortTxIDSelector is these lines, and the pair is what btclib.hashes.siphash takes.
Per message and not per block: the nonce is in the digest, so two senders announcing one block under two nonces produce two sets of short ids, which is what BIP152 asks for – “Nodes SHOULD NOT use the same nonce across multiple different blocks” – so that a collision is one peer’s and not the network’s.
- class btclib.p2p.compact_blocks.GetBlockTxn(block_hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', indexes: Sequence[int] = (), *, check_validity: bool = True)[source]¶
Bases:
PayloadThe getblocktxn message: which transactions of a block are wanted.
BIP152’s BlockTransactionsRequest and Core’s class of that name: the block hash and the indexes, differentially encoded. block_hash is in display order, BlockHeader.hash’s; indexes are absolute, and the module docstring is where that is argued.
What builds one is PartialBlock.missing_indexes, which is the list a reconstruction that came up short answers with – GetBlockTxn( partial.header.hash, partial.missing_indexes) is the whole of it.
An empty indexes is not refused: Core disconnects the peer that sends one – “No legitimate reason to send indexes empty” – which is a rule about a peer and not about a message, and this package holds none of the first kind.
Frozen and hashable, every field being immutable.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) GetBlockTxn[source]¶
Return the absolute indexes the differences name.
- class btclib.p2p.compact_blocks.PartialBlock(header: BlockHeader, transactions: Sequence[Tx | None] = (), *, check_validity: bool = True)[source]¶
Bases:
objectA block with the transactions found so far in it, and gaps for the rest.
What reconstruct answers with, and Core’s PartiallyDownloadedBlock without the word this package cannot use: the header, and one entry per transaction of the block, each either a transaction or None. missing_indexes is what is still wanted and fill is what finishes the block once it has arrived.
Frozen, and not hashable: the entries are mutable Tx objects.
- fill(transactions: Sequence[Tx] = (), *, check_validity: bool = True) Block[source]¶
Return the block, the gaps filled with the transactions supplied.
transactions are a blocktxn’s, in the order missing_indexes asked for them, and there must be exactly as many: a shorter answer leaves a position empty and a longer one names a position nothing asked about, which is what Core’s FillBlock refuses on both sides of its loop.
What is not checked here is that they are the right transactions, and the block is where that shows: a short id collision that survived the pool puts a wrong transaction in a position, and the merkle root Block.assert_valid recomputes is what does not then commit to it – Core reaches for the same answer, calling IsBlockMutated at the end of FillBlock and calling what it catches “Possible Short ID collision”.
check_validity is passed to Block, whose assert_valid is Core’s CheckBlock with mainnet’s target: a block of another network is built with it cleared and asked afterwards, which is the two steps btclib.p2p.data’s BlockPayload asks of a caller too.
- property missing_indexes: list[int]¶
Return the indexes of the transactions still wanted, in order.
Exactly what a getblocktxn names, and in the order a blocktxn answers in: GetBlockTxn(partial.header.hash, partial.missing_indexes) is the request, and fill takes what comes back.
A list rather than an exception, which is the whole of the decision reconstruct owes: a reconstruction that came up short has an answer worth having, and it is this one.
- class btclib.p2p.compact_blocks.PrefilledTransaction(index: int, tx: Tx, *, check_validity: bool = True)[source]¶
Bases:
objectOne transaction a cmpctblock carries whole, and where it belongs.
BIP152’s PrefilledTransaction and Core’s struct of that name: an index and the transaction at it. The sender puts here what it expects the receiver has not got – always the coinbase, which is in no mempool, and “a select few which we expect a peer may be missing”.
index is the absolute index into the block, which is BIP152’s own description of the field; the wire carries the difference from the previous one, minus one, and previous_index is what serialize and parse take that difference against. _NO_PREVIOUS_INDEX is its default and is what makes a standalone one the first of a list: a written zero is index zero. The module docstring is where holding the absolute index rather than the wire’s own is argued.
Not a Payload: no command carries a prefilled transaction on its own, cmpctblock being the message and this a structure inside it.
Frozen, and not hashable: Tx is a mutable dataclass, so the field cannot be hashed and dataclasses.replace is what moves one.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, previous_index: int = -1, *, check_validity: bool = True) PrefilledTransaction[source]¶
Return the transaction and the index the difference names.
The difference is bounded before it is added, and the sum after: both are MAX_BLOCK_TX_INDEX, which is what Core’s DifferenceFormatter::Unser refuses a running index past.
- serialize(previous_index: int = -1, *, check_validity: bool = True) bytes[source]¶
Return the difference from the previous index, then the transaction.
The transaction is written with its witness, BIP152 version 2’s “same format as responses to getdata MSG_WITNESS_TX” – which for a transaction that has no witness is the same octets version 1 would have written, the marker going in only where there is something to mark.
- class btclib.p2p.compact_blocks.SendCmpct(announce: bool = False, version: int = 2, *, check_validity: bool = True)[source]¶
Bases:
PayloadThe sendcmpct message: whether to announce, and in which version.
Bitcoin Core’s msg_sendcmpct: one octet read as a boolean and eight of version, little-endian. announce set is BIP152’s high-bandwidth mode, “the node SHOULD announce new blocks by sending a cmpctblock message”; cleared is the low-bandwidth mode, where blocks are announced with inv or headers and a compact one is asked for.
version defaults to CMPCTBLOCKS_VERSION, which is the encoding this module implements and the only one Core answers to – and the field is written and read unchanged whatever it says, that being what it is for: BIP152 negotiates by each side naming the versions it will speak, so a message naming a version this library does not is a message it must still be able to read.
Frozen and hashable, both fields being immutable.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) SendCmpct[source]¶
Return what the peer announced, the first octet being one or zero.
BIP152: the first integer “SHALL be interpreted as a boolean (and MUST have a value of either 1 or 0)”, so anything else is refused rather than read as true. Core reads the octet through its own bool deserialization, which takes any non-zero value and cannot write back what it read either; refusing is the reading that keeps the octets a caller sends the octets that arrived.
- btclib.p2p.compact_blocks.reconstruct(compact_block: CmpctBlock, pool: Sequence[Tx] = ()) PartialBlock[source]¶
Return the block a compact one and a pool of transactions make.
The prefilled transactions go in the positions they name, and every short id is looked for among the pool; what is left over is PartialBlock.missing_indexes, which is what a getblocktxn asks for. Core’s PartiallyDownloadedBlock::InitData is the same walk, over a mempool and an extra pool rather than over one sequence: which transactions are candidates is the caller’s to decide, this package holding no mempool.
The pool is matched by wtxid, Tx.hash, which is BIP152 version 2’s short id input; the module docstring has why version 1 is not offered and what it would change.
Two refusals, and they are BIP152’s rather than this library’s:
a compact block of no transactions is no block, there being no block without a coinbase. Core’s InitData answers READ_STATUS_INVALID for it;
a compact block whose own short ids are not unique cannot be reconstructed, two positions wanting one transaction, and BIP152’s answer is to ask for the block the ordinary way. Core’s is READ_STATUS_FAILED with “Short ID collision” beside it. It is refused here and not in CmpctBlock.assert_valid, such a message being one a peer legitimately sends.
A pool collision is the third case and is not a refusal: where two different transactions of the pool answer one short id, the position is left missing and is asked for, which is what Core does and why – “eating a round-trip due to FillBlock failure would be annoying”. Taking the first match instead is the bug this shape exists to refuse.
The arguments are checked before any of that, which a free function taking an object a caller already built has to do for itself: a CmpctBlock is what the first has to be and a sequence of Tx the second, so that “not a compact block at all” leaves as this library’s own exception rather than as an AttributeError about a field name. psbt.assert_signatures_only is the precedent, and tests/built_object_contract_test.py the gate over the family.
What is not re-asked is CmpctBlock.assert_valid: a message built with check_validity=False is the caller’s own here as everywhere else in this library. Two of its checks are asked all the same, being what the walk below rests on – _assert_positions, a prefilled index outside the block being an IndexError and not an answer, and the header, which is read for the short id key.
There is no check_validity of its own, and that is because there would be nothing left for it to turn off: this is not a constructor a caller hands fields to, so everything the PartialBlock holds either arrived inside the message or was checked on the way in, and the PartialBlock is therefore built with the flag cleared.
btclib.p2p.data module¶
tx and block: the two messages that deliver what a getdata named.
Bitcoin Core’s msg_tx and msg_block, of test/functional/test_framework/messages.py: one transaction and one block, each serialized exactly as it is serialized anywhere else. So the wire format is btclib.tx’s and btclib.block’s, and what is left for this module is the one thing those two leave open.
`include_witness` is a field of the payload, not an argument of `serialize`. Tx.serialize and Block.serialize take it, and on the wire the answer is the connection’s rather than the transaction’s: BIP144 gives it to the peer that negotiated NODE_WITNESS, and Core answers a getdata for MSG_TX with the stripped encoding and one for MSG_WITNESS_TX with the full one – the same transaction, two messages. Something in this package has to hold that answer, and the three places it could go were:
a field of the payload, which is what is below: a tx message is a transaction and the encoding chosen for it, Payload.serialize keeps the one signature every payload type here has, and to_message needs nothing new;
an argument of `serialize`, which is the encoding supplied at the moment it is written. Refused: Payload.serialize is declared in btclib.p2p.payload because to_message calls it, so an argument here is an argument there too – and then either to_message grows one for every payload type or these two override it, which is the uniformity that module was written to get;
always the witness, on the grounds that MSG_WTX and BIP339 made witness-stripped relay the exception. Refused for what it cannot write: a peer that did not negotiate the witness is answered with the stripped encoding or not at all, so a payload without the flag is a library that cannot serve one.
`parse` answers the flag from the object it just built, and the asymmetry is here rather than left to be discovered. BIP144’s marker and flag say whether a witness rode, and Tx.parse and Block.parse already read them, so is_segwit is that answer without a second reader of the same octets. What it cannot answer is which flag a sender held for a transaction that has no witness: both write the same octets, the marker being written only where there is something to mark, so parse says False and a TxPayload(tx, include_witness=True) over a transaction with no witness is a payload that parses back as False. The encoding round-trips exactly either way, which is the property this package keeps; the object round-trips exactly wherever the wire can tell the two apart, and where it cannot there is nothing to keep.
BIP144’s superfluous witness record – a marker over witnesses that are all empty – is the encoding neither class could reproduce, and neither has to: Tx.parse refuses it where it is read, as Core’s UnserializeTransaction does, so no payload here holds one (issue #1104). It is btclib.tx’s answer and is not repeated here.
The flag is stored as it was given, never reduced against the object. include_witness=True over a transaction with no witness could be recorded as the False it will serialize as, and must not be: Tx and Block are mutable dataclasses, so a flag reduced when the payload was built is a flag that lies the moment a caller signs an input. It is read where it is used, which is where Tx.serialize reads its own.
`TxPayload` and `BlockPayload`, and the suffix is the point. These are the only payload types whose command names a class this library already has, and from btclib.p2p import Tx shadowing btclib.tx.Tx is a collision a caller pays for silently. btclib_node has it – its p2p/messages/data.py declares Tx and Block and imports btclib’s as TxData and BlockData – and it is the reader of the two modules together who cannot then say which is which.
Nothing here bounds a message length. btclib.p2p.limits.MAX_PROTOCOL_MESSAGE_LENGTH is the envelope’s, checked off the header’s length field before a payload is allocated and again in Message.assert_valid, and a block satisfying btclib.block.limits.MAX_BLOCK_WEIGHT is always under it: the weight is 3 * stripped_size + size and the stripped size is at least the header and the transaction count, so size falls short of the cap by three times that. The two constants hold the same number today and are not the same constant, which is the coincidence btclib.p2p.limits exists to keep from becoming an import; a second bound here would be a third place for it to be wrong.
Which encoding answers which request is the caller’s, as it is one layer up: Inventory.is_witness reads BIP144’s bit off a getdata entry and this package holds no policy that turns it into a flag. Nothing here refuses a block whose transactions carry witnesses to a peer that asked for none, because nothing here knows what the peer asked.
- class btclib.p2p.data.BlockPayload(block: Block, include_witness: bool, *, check_validity: bool = True)[source]¶
Bases:
PayloadThe block message: one block, and the encoding chosen for it.
Bitcoin Core’s msg_block, and TxPayload’s two fields over a Block: a getdata for MSG_BLOCK is answered with every witness stripped and one for MSG_WITNESS_BLOCK with them, which is the same block written twice.
Block.assert_valid is CheckBlock, proof of work included and mainnet’s target by default, and it is what assert_valid here asks of the block: a block message of another network is built with check_validity=False and asked afterwards, which is the same two steps Block.assert_valid already asks of a caller holding one.
Frozen, and not hashable, for the reason TxPayload is not.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) BlockPayload[source]¶
Return the block the payload carries, and how it was written.
Block.is_segwit is any transaction carrying a witness, which is exactly when Block.serialize writes a marker: the flag answers for the message the way each transaction’s marker answers for itself.
- class btclib.p2p.data.TxPayload(tx: Tx, include_witness: bool, *, check_validity: bool = True)[source]¶
Bases:
PayloadThe tx message: one transaction, and the encoding chosen for it.
Bitcoin Core’s msg_tx. tx is the transaction and include_witness is what Tx.serialize takes – BIP144’s question, answered by the connection and held here so that Payload.serialize keeps one signature; the module docstring is where that is argued and where what parse can and cannot recover is written down.
include_witness has no default, where Block.serialize’s has one: a message is written for a peer, and which encoding that peer negotiated is not a value this package can pick on its behalf.
Frozen, and not hashable: Tx is a mutable dataclass, so the field cannot be hashed and dataclasses.replace is what re-encodes a payload for another peer.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) TxPayload[source]¶
Return the transaction the payload carries, and how it was written.
Tx.parse reads BIP144’s marker, so is_segwit is what the octets said; the module docstring has what that answer cannot distinguish and why nothing here reads the marker a second time.
btclib.p2p.handshake module¶
The two messages a connection opens with: version and verack.
Bitcoin Core’s msg_version and msg_verack, of test/functional/test_framework/messages.py, read against what net_processing.cpp actually does with the octets – which is not the same thing, and the difference is this module’s one hard decision.
`version`’s trailing fields are conditional, and the conditional is not the protocol version. Core’s test framework reads the relay flag if self.nVersion >= 70001, which is BIP37’s rule as BIP37 states it. Core itself reads it if (!vRecv.empty()), and the same for the user agent and the start height above it. The two disagree about a peer announcing Core’s own protocol version – limits.PROTOCOL_VERSION, the number this module’s own default builds – and stopping after the start height, and Core is the one that decides whether such a peer is accepted: it is.
So the flag’s presence is a field here – relay is None where the octets ended before it – and the protocol version is not consulted. The alternative that reads the version number makes the encoding a function of another field, so that a Version(version=60002, relay=True) is an object with no serialization and a peer at PROTOCOL_VERSION that omitted the flag is a message with no object; the alternative that ignores what is left over breaks this library’s rule that octets are one whole object, and would read two distinct payloads back as the one Version. A field keeps both: every octet is accounted for, whatever follows the flag is refused, and the two payloads are two objects each serializing back to the buffer it came from.
What it costs, and this is the one place it is worth saying. Two things, both of them consequences of a last field that may not be there.
A prefix of one Version encoding is another Version, so the generic “no prefix of an encoding is an object” property that tests/parse_contract_test.py holds every parser to is false of this one by construction. What that property is for – two buffers decoding to one object that serializes back to only one of them – is not: the two buffers are two objects, each writing back the octets it came from. tests/p2p/handshake_test.py drives that, and the exclusion in parse_contract_test.py names it.
And Version.parse takes Octets where every other parser in this package takes BinaryData, because a version payload does not say how long it is: whether the last octet is the relay flag or the first octet of whatever comes next is a question the buffer cannot answer, and the envelope’s length field is what answers it. A caller has that already – Version.parse(message.payload) – so what a stream would add is the one reading that is silently wrong. btclib.bip32.key_origin is the other parser in this library that takes Octets for this reason, and parse_contract_test.py states it there too.
Core’s conditionals reach further up than the relay flag, and this module’s requirements stop where they stop being about a message anybody sends. addr_from through start_height are required here, where net_processing.cpp would accept a version truncated after addr_recv: those if (!vRecv.empty()) are a defence against a short read rather than a statement that five fields are optional, no BIP made any of them so, and no peer above Core’s own MIN_PEER_PROTO_VERSION omits them. The relay flag is the one of them a BIP did make optional, and it is the one modelled as such.
- class btclib.p2p.handshake.Verack(*, check_validity: bool = True)[source]¶
Bases:
PayloadThe verack message: no fields, and an empty payload.
Bitcoin Core’s msg_verack, which serializes to nothing and whose deserialize reads nothing.
A class all the same, and the empty payload is why rather than despite: it is the one payload type whose whole content is its command, so the constant on it is the entire benefit of having a class at all, and it is what proves the shape btclib.p2p.payload.Payload states is uniform. Verack().to_message( magic) is a complete verack, resting on the envelope’s payload=b”” default.
parse refuses an octet, where Core ignores whatever a verack carries – ProcessMessage never reads vRecv for one. This library refuses what follows an object everywhere else, and a verack with a payload is a message that serializes back without it.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) Verack[source]¶
Return a Verack, refusing any octet at all.
Octets are refused by assert_no_trailing, which is where the rule already is; a caller’s stream is left exactly where it was, a verack consuming nothing from one.
- class btclib.p2p.handshake.Version(version: int = 70016, services: int = <ServiceFlags.NODE_NONE: 0>, timestamp: int = 0, addr_recv: NetworkAddress | None = None, addr_from: NetworkAddress | None = None, nonce: int = 0, user_agent: bytes | str | bytearray | memoryview = b'', start_height: int = 0, relay: bool | None = None, *, check_validity: bool = True)[source]¶
Bases:
PayloadThe version message: who is calling, and what it can do.
Nine fields, of which the last may not be on the wire at all. In the order Core serializes them: the protocol version, the service flags, the sender’s clock, the address it is writing to, the address it is writing from, a nonce it recognizes its own connection by, the user agent, the height of its best chain, and BIP37’s relay flag.
relay is True, False or None, and None is not False: it says the octets ended before the flag, which is what a peer older than BIP37 sends and what Core still accepts from any peer. What such a peer means is True – net_processing.cpp initializes bool fRelay = true and overwrites it only where the field is there – and is_relay_requested is that reading, so that a caller does not write if version.relay and answer the opposite of the protocol’s default for every peer that omitted it. The module docstring is why the presence is a field rather than a function of version.
user_agent is octets and not text. Core reads it into a std::string and sanitizes it only for the log – SanitizeString on cleanSubVer – so a peer may put anything at all in it, and decoding here would refuse a message Core accepts. A caller that wants to show one decodes it, with the error handling it wants. MAX_SUBVERSION_LENGTH is the bound on it, Core’s LIMITED_STRING.
timestamp and start_height are signed, version is signed, and services and nonce are not, each following the type Core declares – a negative nTime is what Core clamps to zero on receipt rather than refusing, so it is a value this parses.
addr_recv and addr_from carry no timestamp, which is btclib.p2p.address’s reason for two classes.
version defaults to limits.PROTOCOL_VERSION, Core’s own number and the one a Version built with no argument for it announces – not because this library speaks for a peer’s protocol version, which parse never does, but because a caller building one to send is building this library’s own handshake, and PROTOCOL_VERSION is what that is.
- property is_relay_requested: bool¶
Answer whether this peer wants transactions announced to it.
BIP37’s flag, with BIP37’s default where the flag is absent: Bitcoin Core’s net_processing.cpp declares bool fRelay = true before it reads the message and assigns to it only inside if (!vRecv.empty()), so a version that stops before the flag asks for relay rather than refusing it.
The reading and not the field, which is what relay is: a caller that needs to know whether the peer said so reads relay is None. Reading relay itself for the answer is what makes an absent flag mean the opposite of what the protocol says it means.
- classmethod parse(data: bytes | str | bytearray | memoryview, *, check_validity: bool = True) Version[source]¶
Return the version the payload describes, relay flag or not.
The flag is read where an octet is left and left None where none is, which is the one conditional here; everything before it is required, and the module docstring is why.
Octets and not BinaryData, which is the other half of that decision: “where an octet is left” is a question about the whole payload, and in a stream holding the next message the answer would be the first octet of that one. The envelope is what says where a payload ends, so message.payload is what this takes.
Only 0x00 and 0x01 are a flag. Core’s Unserialize for a bool takes any octet and answers != 0, so 0x02 reads as true there and is written back as 0x01 – two payloads, one object, and only one of them serialized back. That is the malleability Message’s command padding is refused for one layer down, and the same answer is given here.
btclib.p2p.inventory module¶
What a peer has, what it wants, and what it could not find.
The messages by which peers tell each other what they have and ask for what they do not, and the type code under them. inv, getdata and notfound are a vector of Bitcoin Core’s CInv, of src/protocol.h; getblocks and getheaders are a protocol version, a locator and a stop hash; headers is block headers. The layout is Core’s test/functional/test_framework/messages.py – msg_inv, msg_getdata, msg_notfound, msg_getblocks, msg_getheaders, msg_headers and the CInv and CBlockLocator under them – and the type codes are src/protocol.h’s GetDataMsg.
Three commands share one body, and so one class each over one private base, which is btclib.p2p.keepalive’s shape for ping and pong: Inv, GetData and NotFound are one vector of Inventory, GetBlocks and GetHeaders one (version, locator, stop hash). A field naming the command instead would let a caller build an inv that serializes under “getdata”; a subclass setting command cannot, and the generated __eq__ keeps an inv and a getdata of the same octets two objects.
`InventoryType` is an `IntEnum`, and the `ServiceFlags` reasoning does not transfer. Eight octets of service flags are a bitfield throughout, so IntFlag is what the field is and an unnamed bit is a service not yet heard of. A type code is not a bitfield: MSG_TX, MSG_BLOCK, MSG_FILTERED_BLOCK, MSG_CMPCT_BLOCK and MSG_WTX are one through five, exclusive kinds and not bits, and only MSG_WITNESS_FLAG is a bit. An IntFlag over them would compose nonsense and answer for it: 1 | 4 would be MSG_WTX, so MSG_TX in InventoryType(5) would be true and a wtx announcement would test as a tx one. IntEnum is what says these are kinds – and the composites are members because Core makes them members, MSG_WITNESS_TX and MSG_WITNESS_BLOCK being named in GetDataMsg itself. MSG_FILTERED_BLOCK | MSG_WITNESS_FLAG is not: BIP144 reserved it and Core carries it commented out, so naming it here would be btclib publishing a code Core does not.
What the flag being a bit still buys is Inventory.is_witness, which is the reading rather than the field, as Version.is_relay_requested is: a caller asking whether a peer wants the witness reads that instead of comparing against two members and forgetting the third the next BIP adds.
An unrecognized type code round-trips, which is the envelope’s rule about an unrecognized command one layer down. IntEnum refuses a value no member names, so the coercion is _inventory_type_from_int and it hands back the plain int where there is no member – exactly what _inventory_type_from_int’s neighbour _service_flags_from_int does for what is no bitfield at all. Inventory.type_code is therefore InventoryType | int, four octets wide either way, and the octets a peer sent come back as they arrived.
`headers` carries a transaction count that is always zero, and it is dropped rather than stored. Core writes a header followed by an empty transaction vector – msg_headers.serialize builds a CBlock per header for exactly that – and reads it as ReadCompactSize(vRecv); // ignore tx count; assume it is 0. Storing it would be a second object for one meaning: a headers message says nothing about how many transactions a block has, so a field holding what a peer wrote there would be a field with no reading. What dropping it costs is an encoding this cannot reproduce, so a non-zero count is refused rather than ignored, and the property kept is the one this library keeps everywhere: every payload it accepts serializes back to the octets it came from. Verack.parse refuses an octet Core ignores for the same reason, and Version.parse refuses a relay flag of 0x02 that Core reads as true. A non-minimal encoding of the zero needs no refusal of its own, btclib’s var_int.parse being canonical-only already.
`getblocks` and `getheaders` carry an ignored field too, and that one is stored, which is the contrast worth having in one place. Core writes the stream’s protocol version in front of the locator and discards what it reads – CBlockLocator’s SERIALIZE_METHODS on one side, msg_getblocks on the other, whose comment reads “Bitcoin Core ignores the version field. Set it to 0.” Ignored is not constant: a Core node sends the version it negotiated, its test framework sends zero, and a message carrying either is one both accept. So the field varies over messages anybody sends and is a field; the headers count does not vary at all.
Every hash is held the way a block explorer prints it and reversed on the wire, as OutPoint.tx_id and BlockHeader.previous_block_hash are: these are the same hashes those fields hold, so a caller comparing an announcement with a transaction it has must not have to reverse one of them. tests/p2p/inventory_test.py is driven by the vendored mainnet blocks for that reason – the hash of a real block is what tells the two orders apart, and nothing a round trip does can.
Every count is bounded before the loop that allocates on it, each under Core’s own name in btclib.p2p.limits: MAX_INV_SZ, MAX_HEADERS_RESULTS, MAX_LOCATOR_SZ. A count is the peer’s to choose and btclib’s var_int.parse allows 33,554,432 of anything, so the check before the loop is the whole of what a bound is for – and, as in Addr.parse, it does not answer to check_validity, a defence a caller can turn off not being one.
- class btclib.p2p.inventory.GetBlocks(version: int = 0, locator: Sequence[bytes | str | bytearray | memoryview] = (), hash_stop: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', *, check_validity: bool = True)[source]¶
Bases:
_LocatorPayloadThe getblocks message: announce the blocks after this point.
Bitcoin Core’s msg_getblocks. The answer is an inv of up to five hundred block hashes, which the asker then fetches with a getdata – one round trip more than getheaders, and what a node without the headers-first sync uses.
- class btclib.p2p.inventory.GetData(items: Sequence[Inventory] = (), *, check_validity: bool = True)[source]¶
Bases:
_InventoryPayloadThe getdata message: send me these, by identifier.
Bitcoin Core’s msg_getdata, and the one of the three whose codes use the full vocabulary: MSG_FILTERED_BLOCK, MSG_CMPCT_BLOCK and the witness composites “can only occur in getdata”, which is where Inventory.is_witness earns its place.
- class btclib.p2p.inventory.GetHeaders(version: int = 0, locator: Sequence[bytes | str | bytearray | memoryview] = (), hash_stop: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', *, check_validity: bool = True)[source]¶
Bases:
_LocatorPayloadThe getheaders message: send the headers after this point.
Bitcoin Core’s msg_getheaders. The answer is a headers message of up to MAX_HEADERS_RESULTS headers, and a shorter one is how the asker learns it has reached the peer’s tip – Core’s comment on that constant says so, and calls changing it a protocol upgrade.
- class btclib.p2p.inventory.Headers(headers: Sequence[BlockHeader] = (), *, check_validity: bool = True)[source]¶
Bases:
PayloadThe headers message: block headers, and no transactions.
Bitcoin Core’s msg_headers: a count, and that many eighty-octet headers each followed by a transaction count of zero. The count is not a field of this class and the module docstring is why; serialize writes the zero after every header and parse refuses anything else.
BlockHeader is the type, btclib.block already parsing and serializing one: a header vector is that class in a loop rather than a second reader of the same eighty octets. Which is also what makes the elements answerable to assert_valid_pow – this class does not call it, a headers message being how a node learns of work it has not checked, and the caller is who decides when to.
`MAX_HEADERS_RESULTS` bounds the count before the loop, Core’s own bound on this message and the reason its handler reads the headers by hand: “we don’t want to risk deserializing 2000 full blocks”.
Frozen, and the one class here that is not hashable: BlockHeader is a mutable dataclass, so a tuple of them cannot be hashed. dataclasses.replace is what changes the headers in one.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) Headers[source]¶
Return the headers the payload carries, the count bounded first.
The transaction count after each header is read and refused unless it is zero, where Core reads it and throws it away: a headers carrying any other number is a message this library could not write back, and refusing what it cannot reproduce is what it does everywhere else – Verack.parse on a payload Core ignores, Version.parse on a relay flag Core accepts.
- class btclib.p2p.inventory.Inv(items: Sequence[Inventory] = (), *, check_validity: bool = True)[source]¶
Bases:
_InventoryPayloadThe inv message: what this node has, offered unasked.
Bitcoin Core’s msg_inv. An announcement and not a delivery: what a peer does with one is send back a getdata for the entries it wants, which is the next class down.
Core announces transactions as MSG_TX or MSG_WTX and blocks as MSG_BLOCK – “Invs always use TX/WTX or BLOCK”, says the comment in GetDataMsg – and the witness bit belongs to a getdata. That is a rule about what a node sends, not about what these octets can carry, so nothing here refuses the other codes: a peer that sends one is answered by the caller’s own policy, this package holding none.
- class btclib.p2p.inventory.Inventory(type_code: int = InventoryType.UNDEFINED, hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', *, check_validity: bool = True)[source]¶
Bases:
objectOne entry of an inv, a getdata or a notfound: (type, hash).
Bitcoin Core’s CInv, of src/protocol.h, whose SERIALIZE_METHODS is READWRITE(obj.type, obj.hash): four octets of type code, little-endian, and the thirty-two of a hash256.
type_code is an InventoryType where a member names the code and the plain int where none does, which is how a code this library has not heard of round-trips. Core spells the field type and this library cannot: a class attribute of that name shadows the builtin inside its own body, and type[Inventory] is what every parse here annotates its cls with – “type code” being what the protocol documentation calls it in prose anyway.
hash is a transaction id, a witness transaction id or a block hash depending on that code, and is held in the order a block explorer prints: the order OutPoint.tx_id and BlockHeader.previous_block_hash are held in, these being those very hashes.
Frozen and hashable, both fields being immutable: an entry is a value, and it is what a set of announcements from a peer holds.
- property is_witness: bool¶
Answer whether the code asks for the witness beside the data.
BIP144’s bit, MSG_WITNESS_FLAG, which Core ORs into a type code rather than listing beside it – GetFetchFlags is the one line that sets it, on a peer that may be served witnesses.
The reading and not the field, as Version.is_relay_requested is: comparing type_code against MSG_WITNESS_TX and MSG_WITNESS_BLOCK is the same question asked in a way that goes wrong the day a BIP names a third composite.
- class btclib.p2p.inventory.InventoryType(*values)[source]¶
Bases:
IntEnumWhat an inventory entry’s hash identifies, Core’s GetDataMsg.
src/protocol.h, spelled as Core spells it, composites included: MSG_WITNESS_TX and MSG_WITNESS_BLOCK are members there rather than something a caller assembles, so they are members here. The third BIP144 reserved, MSG_FILTERED_BLOCK | MSG_WITNESS_FLAG, is not: Core’s own line for it is commented out as “reserved for future use and remains unused”, and a library naming it would be publishing a code the protocol has not got.
UNDEFINED is Core’s name for zero, which is the code an entry carries when it identifies nothing – CInv’s default, and what its test framework’s type map calls “Error”.
An IntEnum and not an IntFlag: the module docstring has why the ServiceFlags reasoning stops here, and _inventory_type_from_int is what keeps a code no member names from being an error.
GetDataMsg is Core’s name and is not taken here, being the name of one of the three messages that carry the code: an inv announces with it and a notfound refuses with it.
- class btclib.p2p.inventory.NotFound(items: Sequence[Inventory] = (), *, check_validity: bool = True)[source]¶
Bases:
_InventoryPayloadThe notfound message: I have none of these.
Bitcoin Core’s msg_notfound, the answer to a getdata naming something this node cannot serve – a transaction that has left its mempool, most often.
`MAX_INV_SZ` bounds this one too, where Core does not bound it, and the difference is worth stating rather than glossing: Core refuses an over-long inv or getdata with a Misbehaving and answers an over-long notfound by ignoring its contents instead. A parser has to bound it all the same – a count is the peer’s to choose, and this is the same loop the other two run – and MAX_INV_SZ is the right number for it because a notfound answers a getdata, which cannot have held more.
btclib.p2p.keepalive module¶
ping and pong: a nonce out, and the same nonce back.
BIP31’s two messages, and Bitcoin Core’s msg_ping and msg_pong: eight octets of nonce each, little-endian, and net_processing.cpp answers a ping by writing the nonce it read straight back. The nonce is what tells one round trip from the next – BIP31’s own reason, quoted in Core’s comment: without it a peer that pings every second and is answered after five cannot tell which answer belongs to which question.
The pre-BIP31 `ping` is not modelled, and that is a decision rather than an omission. Before protocol version 60001 a ping had no payload at all, and Core still accepts one – if (pfrom.GetCommonVersion() > BIP0031_VERSION) is what guards the read. Nothing has sent one since 2012, and a nonce of int | None would put that on every caller of pong.nonce forever. What answers such a peer is the envelope, which round-trips a ping with an empty payload as Message(magic, “ping”) without this module being involved – an unrecognized command and an unmodelled shape of a recognized one are the same thing there, on purpose.
pong has no such history: it is BIP31’s, so it has carried a nonce from the message’s first version. Core reads it defensively all the same – ProcessPong checks nAvail >= sizeof(nonce) and treats a short one as a peer misbehaving rather than as a message – which is a policy about what to do with octets that do not decode, and this package holds none.
- class btclib.p2p.keepalive.Ping(nonce: int = 0, *, check_validity: bool = True)[source]¶
Bases:
_NoncePayloadThe ping message: a nonce a pong is expected to echo back.
A nonce of zero is a nonce, which is worth saying because it is the one an implementation reading its own field for truth invents a replacement for – and Core does send zero, ProcessMessage declaring uint64_t nonce = 0 and writing back whatever it read. There is no default nonce here beyond the field’s own: choosing one is drawing a random number, which is the caller’s and secrets’.
- class btclib.p2p.keepalive.Pong(nonce: int = 0, *, check_validity: bool = True)[source]¶
Bases:
_NoncePayloadThe pong message: the nonce of the ping it answers.
Whether it is the nonce that was sent is the caller’s question and not this codec’s: matching an answer to a question is what a connection does, and this package holds no connection.
btclib.p2p.limits module¶
The protocol limits on a p2p message, with Bitcoin Core’s names.
A module of its own, as block/limits.py and script/limits.py are and for the same reason: the other modules here are the dataclasses and their serializations, while the names here are rules about a message being accepted, and a caller naming this module is saying the numbers are Core’s rather than this library’s.
Core declares them among the constants of peer management and not among the consensus ones, which is where they belong: nothing about a block or a transaction changes with them, and what each bounds is what a node allocates for a count or a length a peer chose. Not one of them is a consensus rule, so none of them is in btclib.consensus – and MAX_PROTOCOL_MESSAGE_LENGTH is 4,000,000 as consensus.MAX_BLOCK_WEIGHT is, which is the coincidence this module exists to keep from becoming an import.
An envelope without MAX_PROTOCOL_MESSAGE_LENGTH is what btclib_node has – verify_headers reads the four octets and waits for 24 + payload_len with nothing between the peer’s number and the buffer – and the bound is the difference between refusing such a header at once and holding whatever the peer dribbles in against a length it will never reach. Every count bound here has the same shape one layer up: parse reads the count before it builds anything, where btclib_node’s own message classes loop over whatever var_int.parse allowed them, and btclib’s var_int cap is 33,554,432.
Not every name here is checked by something, and BIP157’s are where the difference shows: a bound on a range whose far end is a block hash cannot be applied without the chain that turns the hash into a height, so MAX_GETCFILTERS_SIZE is published for the caller holding one and checked nowhere. Publishing it is what keeps such a caller from writing the number down a second time.
btclib.p2p.magic module¶
Where the p2p message start comes from, which btclib does not hold.
btclib/network.py decided this before there was a p2p package, and its Network docstring is where the decision is: “No consensus parameter lives here, and no p2p one: the message start belongs to the code that speaks to a node, bitcoin_core_rpc.magic_from_chain being where it is, because a custom signet’s is a function of its challenge and therefore not a field any table can hold.”
So the question this module answers is not which of two designs to choose. Giving Network a magic field would contradict a docstring that states its own reason, and the reason is the fifth network: NETWORKS is an encoding table fixed at import, every field of which is the same for every deployment of the network it describes, while a custom signet’s message start is the first four octets of the sha256d of its block challenge and differs between two deployments that report the same chain. A field would be right for four networks and a lie for the fifth, which is an annotation accepting the mistake rather than refusing it. btclib.fetch.transport re-exports the same package’s HTTP transport on the same reasoning: a second copy is a second thing to keep true.
magic_from_chain and magic_from_signet_challenge are therefore aliases and not wrappers – the package’s own objects, so btclib.p2p.magic_from_chain is bitcoin_core_rpc.magic_from_chain – and they take Core’s vocabulary: a chain name, “main”, “test”, “testnet4”, “signet” or “regtest”, and a challenge as the hex a config file writes or the octets a parser holds. Their exceptions are the package’s too, as btclib.fetch.transport says of the transport it re-exports: an unknown chain leaves as BtcRpcValueError, which is not a class btclib.exceptions declares.
magic_from_network is what btclib’s own vocabulary reaches, and it is here because that vocabulary is not Core’s: NETWORKS is keyed by the BIP network names – “mainnet”, “testnet” – where Core says “main” and “test”, so a caller holding a Network and passing its name straight to magic_from_chain is told its network is unknown. bitcoin_core_rpc.chain_from_network is the bridge, and btclib.fetch.bitcoin_core uses it exactly this way: “chain_from_network on the way in, client_errors on the way out”.
On the way out there is nothing to translate, and that is by construction rather than by luck. network._validated_network_name is what the name goes through first – the strip().lower() tolerance issue #216 decided every network: str parameter keeps – so what reaches chain_from_network is a key of NETWORKS, and every one of those is a chain Core has. What could raise the package’s class is therefore refused before the call, by btclib’s own converter and with btclib’s own exceptions; a try/except around the call would be a second check of what the first has already settled, which is the guard this tree does not write. tests/p2p/magic_test.py walks NETWORKS and asserts the property instead, which is where it can go stale visibly.
btclib.p2p.message module¶
The Message dataclass; the class docstring has the contract.
The layout is Bitcoin Core’s CMessageHeader, of src/protocol.h: four octets of message start, twelve of message type, four of payload size little-endian and four of checksum, followed by the payload the last two describe. Core’s own test framework writes the same header by hand, in test/functional/test_framework/p2p.py.
Core spells the second field m_msg_type, where the wire documentation, the BIPs and every other implementation say “command”. The name here is the one a reader of the protocol meets; the citations are Core’s, so both spellings are worth having in one place.
- class btclib.p2p.message.Message(magic: bytes | str | bytearray | memoryview, command: bytes | str | bytearray | memoryview, payload: bytes | str | bytearray | memoryview = b'', *, check_validity: bool = True)[source]¶
Bases:
objectOne p2p message: which network it is for, what it is, and its payload.
Three fields, where the header has four and the payload follows it, because two of the four are not the object’s to hold. The checksum is the first _CHECKSUM_SIZE octets of hash256(payload) and is a property, so that no instance can carry one disagreeing with the payload beside it; the payload length is len(payload) for the same reason. What is left – the magic, the command and the payload – is what the octets do not determine.
magic is four octets and is looked up nowhere: an unfamiliar one round-trips, a custom signet’s being derived from its challenge rather than tabulated – btclib.p2p.magic is where a caller gets one and where that decision is argued.
It is a field of the message and not something a connection puts in front of it, which is where this departs from btclib_node: there messages.add_headers writes the other three fields and p2p.connection.Connection._send prepends the magic, so what the codec serializes is not a message and is not what its own verify_headers reads back – that one indexes the length at 16, which is where it sits once the magic is there. One header across two layers is also the one shape unavailable to a package that holds no connection.
command is the message type as text, “version” or “verack”, without the NUL padding the wire puts after it, and an unknown one round-trips as opaque bytes do: this class is the envelope, and it knows no payload type (issue #1083).
Text and not the twelve octets verbatim, which is the choice a round-trip is usually the argument against: strip the padding and two wire values decode to one object, which is the malleability assert_no_trailing is spent on one field down. What answers it is not keeping the octets but refusing the ones Core refuses – _command_from_bytes is IsMessageTypeValid – so the twelve octets and the text are one to one in both directions, and every value this accepts serializes back to the value it was read from. Keeping the field verbatim would do the opposite of what it looks like: it would round-trip a “ping” with a stray octet after its NUL faithfully, and that is a header Core drops the sender for, so btclib would be reading and re-emitting a message no peer accepts. It would also put the padding in every caller’s hands, where String and str_from_string are what the rest of this library uses for a field that is ascii text.
Frozen, all three fields being immutable: a message is a value, and dataclasses.replace is what retargets one at another network.
No to_dict and no from_dict, where the other wire-format classes of this library have both: those agree with a json shape somebody else writes too – Core’s rpc for a transaction, BIP174’s for a psbt – and nothing renders a p2p envelope as json, so the pair would be inventing a shape rather than reading one, over a payload that is opaque octets either way.
- assert_valid() None[source]¶
Refuse a magic, a command or a payload no message carries.
The payload bound is MAX_PROTOCOL_MESSAGE_LENGTH: a message above it is one no peer accepts, so serializing one would be writing octets with nowhere to go. parse refuses the same bound and it is not the same check – there it is read off the length field before the payload is allocated, and it cannot be turned off.
- property checksum: bytes¶
Return the four octets the header carries.
The first four octets of hash256(payload), Bitcoin Core’s V1Transport::GetMessageHash. It is what the payload says it is, so it is derived here and never stored; parse is the one place the two can disagree, and it refuses the octets rather than building the disagreement.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) Message[source]¶
Return one message, telling “not all there yet” from “never”.
A BytesIO is the caller’s stream and may hold part of the next message, or several whole ones: what is consumed is this message, and the stream is left on the octet after it, which is how a caller reading off a socket takes them one at a time – the position being what says how much was consumed. Octets are one whole object and what follows the message in them is refused, as everywhere in this library; btclib.utils states both halves.
Octets that end inside the message raise `IncompleteMessageError`, and the stream is rewound to where the message started. It is the one refusal here that more octets can answer, so the caller reads missing more, appends them and calls again on a stream still positioned at the start. Every other refusal is final and none of them rewinds: a peer whose header does not decode is a peer Core disconnects rather than resynchronizes with, and what to do about it is the caller’s policy, not this package’s.
The header is read as one unit rather than field by field, which is Bitcoin Core’s own split – V1Transport has a header phase and a body phase – and is what makes missing exact: the octets still wanted are the rest of the header, and once the header is in hand, the rest of the payload. A field-by-field read can only report what the field it stopped in was short of, which is not a number the caller can ask its socket for.
The payload length is compared with MAX_PROTOCOL_MESSAGE_LENGTH before the payload is asked for: the field is the peer’s to choose, and the whole of what the bound is for is that nothing allocates on it first. That check does not answer to check_validity – a defence a caller can turn off is not one – and neither does the checksum, which is what tells this payload from the octets a link corrupted: skipped, two buffers would decode to the one object that serializes back to only one of them.
The magic is read and not checked, no argument here naming the network expected. A caller that means to refuse another chain’s message compares message.magic with what it expects, which is the one line an optional magic=None here would replace with a defence that is off unless asked for; and a magic no table holds is a custom signet’s, which must round-trip rather than be refused.
btclib.p2p.negotiation module¶
The messages by which a peer says what it wants sent to it.
getaddr wants addresses, mempool wants what is in the mempool, sendheaders wants a new block announced as a header rather than as an inventory, wtxidrelay wants transactions announced by wtxid, sendtxrcncl says this peer would reconcile its transaction set rather than have every one announced, feefilter wants nothing below a fee rate, and feature names a feature whose messages may then be sent to it. That is the one idea they share, and the module is named for the larger half of it: getaddr and mempool are one-off requests, and the rest negotiate how the connection is written from there on. Issue #1119 is titled for both – “the p2p negotiation and request messages” – and a filename cannot be.
Why they are together, and it is not that they answer to nothing. Every other request in this package sits beside the message that answers it: getdata beside inv, getcfilters beside cfilter, getblocktxn beside blocktxn. The two requests here cannot follow that rule. A getaddr is answered by an addr or by an addrv2, which are two modules, so sitting beside its answer means picking one of them; a mempool is answered by an inv, and inventory is the module about inventories rather than about what a peer holds. Those that negotiate have the opposite problem: they turn nothing on that this package encodes, so there is no codec for them to sit beside at all.
sendaddrv2 and sendcmpct are not here for the reason the negotiators are: each is about a message this package does encode – an addrv2 and a cmpctblock – so each lives beside the codec it turns on, which is where a reader of that codec looks.
`getaddr`, `mempool`, `sendheaders` and `wtxidrelay` carry no octets at all, which makes them the shape Verack and SendAddrV2 already are, down to the one thing an empty payload has to get right: parse refuses trailing octets rather than ignoring them, a message this class could not have written not being one it may answer. They repeat those lines rather than share a base, for the reason addrv2.SendAddrV2 states: keepalive._NoncePayload is a base because two commands have one body, and the absence of a body is not a body to share. The alternative – one class with command as a field – is refused across this package and argued in payload.py: the command is the message’s identity, and a field would let a caller build a getaddr that serializes under “mempool”.
`feature` is the general case of the negotiators beside it. BIP434 generalises what sendaddrv2 and wtxidrelay each do with a command of their own: one message carries a featureid naming the feature and a featuredata holding whatever that feature’s own specification puts there, so that the next feature needs neither a command minted for it nor a protocol version number agreed on – “there is no longer a question whether version ‘n+1’ belongs to Alice’s new feature, or Bob’s new feature”.
What a featureid names is therefore not this package’s to know, and there is no table of them here. BIP434 has a node “ignore feature messages specifying a featureid they do not support, so long as the payload conforms to the requirements above”, which makes the identifier something a caller matches on and this codec merely carries – the same if payload.py leaves it for a command.
`feefilter` is a signed `int64_t`, and signed is not an accident of the type Core happened to declare. BIP133 defines the message as one “containing an int64_t”, to be “interpreted as satoshis per kilobyte”; it states the type and the units and not the encoding. The octets are the Bitcoin Wiki’s Protocol documentation, which publishes the payload as eight bytes of that integer, LSB first, cited by revision as the envelope’s tests cite it: https://en.bitcoin.it/w/index.php?title=Protocol_documentation&oldid=68832
Core reads it into a CAmount – which is int64_t – and asks MoneyRange(newFeeFilter) only after the read. So a negative fee rate is octets Core parses and declines to use, not octets it refuses, and this codec parses them too: the money range is policy about a value, and this package holds no policy. What it does refuse is a value no eight octets hold, which is the field boundary rather than a judgement.
`sendtxrcncl` is BIP330’s Erlay negotiation and none of Erlay’s reconciliation. The message itself is two unsigned fields, a uint32 protocol version and a uint64 salt; BIP330’s own table and Core’s test/functional/test_framework/messages.py msg_sendtxrcncl agree on that layout field for field. node/txreconciliation.h declares TXRECONCILIATION_VERSION 1, matching BIP330’s “Sender must set this to 1 currently” – so both sources name the same one value a peer sends today, and this codec fixes neither: a version below 1 is a protocol violation BIP330 states and Core enforces in TxReconciliationTracker::RegisterPeer, after the parse and against the pair of versions the two peers offered, which is a connection’s state and not a fact the four octets of version carry alone; a salt is entropy, and every value its width holds is one a peer may have picked.
Erlay itself – the sketches, the Minisketch library, the request-and-response round that follows a successful negotiation – is not modelled here and is not going to be. BIP330’s own list of the new messages the protocol adds is sendtxrcncl and four more – reqrecon, sketch, reqsketchext, reconcildiff – and this module carries the first and none of the rest; carrying it and stopping is the shape every other module in this package already has, codecs and no behaviour, stated once so that the absence of the other four reads as the boundary it is rather than as an unfinished job. Issue #1066 is what decides it and how: “what btclib can build out of the Python standard library is in scope, what would need a hand-rolled [construction] is not” – hashlib and hmac back HKDF, and nothing in the standard library backs a PinSketch over GF(2**32). Minisketch is C++ behind a C API for the reason btclib_secp256k1 wraps a C library, and a pure-Python sketch would be the only implementation on a relay path rather than a documented slow arm behind a fast default, which is the asymmetry issue #1066 states for a cipher and which holds here too.
What is not modelled is placement, and the rules are not the same for all of them. wtxidrelay must arrive before the verack and Core disconnects a peer that sends one after it; sendheaders has no such rule and Core honours one whenever it arrives; getaddr is answered once per connection and only to an inbound peer; mempool is served only where Core advertises NODE_BLOOM or the peer is permitted; Core sends a feefilter from protocol version 70013, unless it is running with transactions ignored, the peer has the force-relay permission, or the connection is block-relay-only; and sendtxrcncl, like wtxidrelay, belongs between version and verack – BIP330 says a peer that sends one after verack should be disconnected, and Core’s net_processing.cpp does exactly that, along with disconnecting a peer that offers reconciliation while either side’s version declined transaction relay; and BIP434 forbids a feature after the verack and to a peer advertising a protocol version below 70017, requiring one that arrives between the version and the verack be accepted. Every one of those is a rule about when or to whom, which needs a connection to hold, and this package has none – the same line SendAddrV2 draws for BIP155’s placement rule.
- class btclib.p2p.negotiation.Feature(feature_id: bytes | str | bytearray | memoryview, feature_data: bytes | str | bytearray | memoryview = b'', *, check_validity: bool = True)[source]¶
Bases:
PayloadThe feature message: a feature this peer supports, and its data.
BIP434, and Bitcoin Core’s msg_feature: a featureid and a featuredata, each a length and that many octets. The identifier is what the feature is negotiated under – the BIP number for one published as a BIP, “some other unique identifier” such as a URL or a digest for one that is not – and the data is whatever that feature’s own specification puts there, empty where it wants none.
feature_id has no default, as Message’s command and CmpctBlock’s header have none: an identifier is the whole of what this message says, so an object without one could not be valid.
feature_id is octets and not text, as Version.user_agent is. BIP434 asks for printable ASCII with a SHOULD rather than a MUST, and Core’s p2p_bip434_feature.py asserts a node accepts an identifier that is not – test_non_ascii_feature_id_accepted – so decoding here would refuse a message Core answers. A caller that wants to show one decodes it, with the error handling it wants.
Both lengths are refused rather than parsed, which is the opposite of what FeeFilter does with the money range: BIP434 writes them as a MUST on the encoding and Core answers a payload outside them with a disconnect, where MoneyRange is asked about a value already read. btclib.p2p.limits holds the numbers and the citation.
Frozen and hashable, both fields being immutable.
- class btclib.p2p.negotiation.FeeFilter(feerate: int = 0, *, check_validity: bool = True)[source]¶
Bases:
PayloadThe feefilter message: do not announce below this fee rate.
BIP133, and Bitcoin Core’s msg_feefilter: eight octets of a fee rate, little-endian, “interpreted as satoshis per kilobyte”, and a peer that sends one is asking not to be told about a transaction paying less. It is a request and not a rule, in BIP133’s own words: the receiving node “will be permitted, but not required, to filter transaction invs for transactions that fall below the feerate provided”. The one thing the BIP says about the filter’s reach is that it does not stop at newly relayed transactions – “Inv’s generated from a mempool message are also subject to a fee filter if it exists”.
The rate is signed, CAmount being int64_t, and a value outside the money range is parsed rather than refused: Core asks MoneyRange before it acts on one, which is what to do with a value rather than whether the octets decode. The module docstring is the whole of that argument.
Frozen and hashable, the one field being immutable.
- class btclib.p2p.negotiation.GetAddr(*, check_validity: bool = True)[source]¶
Bases:
PayloadThe getaddr message: send me the peers you know of.
Bitcoin Core’s msg_getaddr, and the whole of that command: the answer is an addr, or an addrv2 where the peer asked for one.
Core answers at most once per connection and only an inbound peer – ProcessMessage returns early on pfrom.IsInboundConn() false and on peer->m_getaddr_recvd – and it caps and shuffles what it sends. All three are policy a connection carries out, and none of them is visible in the octets, which is why none of them is here.
- class btclib.p2p.negotiation.Mempool(*, check_validity: bool = True)[source]¶
Bases:
PayloadThe mempool message: send me what your mempool holds.
BIP35, and Bitcoin Core’s msg_mempool: the answer is an inv of the transactions in the peer’s mempool, or several, bounded the way any inv is.
Core serves one only where it advertises NODE_BLOOM itself, or where the asking peer holds the mempool permission – the check is on the answering node’s own services and not on the asker’s – and it drops a peer that asks otherwise unless that peer may not be banned. That is policy about who may ask and about what this node offers; the octets of the question are the same either way.
- class btclib.p2p.negotiation.SendHeaders(*, check_validity: bool = True)[source]¶
Bases:
PayloadThe sendheaders message: announce a new block as a header.
BIP130, and Bitcoin Core’s msg_sendheaders: a peer that sends one is asking to be told about a new block with a headers message rather than with an inv it would then have to ask about. The saving is the round trip, which is BIP130’s own argument.
There is no message that turns it off again, and no rule that it be honoured either. BIP130 is permissive in both directions: the receiving node “will be permitted, but not required, to announce new blocks by sending the header”, and implementations “may also optionally impose additional constraints, such as only honoring sendheaders messages shortly after a connection is established”. Core imposes none of them – m_prefers_headers = true is the only write there is, and it happens whenever the message arrives.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) SendHeaders[source]¶
Return a SendHeaders, refusing any octet at all.
- class btclib.p2p.negotiation.SendTxRcncl(version: int = 0, salt: int = 0, *, check_validity: bool = True)[source]¶
Bases:
PayloadThe sendtxrcncl message: this peer would reconcile, not announce.
BIP330, and Bitcoin Core’s msg_sendtxrcncl: a uint32 protocol version and a uint64 salt, this peer’s half of the entropy the two sides combine – TaggedHash(“Tx Relay Salting”, salt1, salt2), the lower salt first – to key the short transaction IDs a reconciliation round exchanges. The module docstring has the whole of what this message is the negotiation for and is not the codec of.
version is 1 for every peer running the protocol BIP330 and node/txreconciliation.h describe today, and this class does not enforce that: a version below 1 is what Core’s TxReconciliationTracker::RegisterPeer calls a protocol violation, after comparing the two peers’ versions against each other, which this codec never sees. salt is entropy, and every value its width holds is one a peer may have chosen.
Frozen and hashable, both fields being immutable.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) SendTxRcncl[source]¶
Return the version and the salt the twelve octets carry.
- class btclib.p2p.negotiation.WtxidRelay(*, check_validity: bool = True)[source]¶
Bases:
PayloadThe wtxidrelay message: announce transactions by wtxid.
BIP339, and Bitcoin Core’s msg_wtxidrelay: a peer that sends one is asking for MSG_WTX in the inventories it is sent, so that a transaction is named by the hash that commits to its witness and a witness-malleated copy is a different announcement rather than the same one.
One message enables one direction. BIP339: “After a node has received a wtxidrelay message from a peer, the node MUST use the MSG_WTX inv type when announcing transactions to that peer” – so a peer that sends one has said how it wants to be announced to, and said nothing about how it will announce. Core matches, setting m_wtxid_relay on receipt alone and never consulting whether it sent its own; two Core nodes both send one, which makes the connection symmetric in practice and is a fact about Core rather than about the message.
Like BIP155’s sendaddrv2, it belongs between the version and the verack, and Core disconnects a peer that sends one after. That is a rule about when, and holding it needs a connection this package does not have.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) WtxidRelay[source]¶
Return a WtxidRelay, refusing any octet at all.
btclib.p2p.payload module¶
How a payload type meets Message, decided here for every payload.
btclib.p2p.message knows no payload type and reads a command it has never heard of exactly as it reads “version” (issue #1082). This module is the other side of that: a payload is an ordinary btclib wire class – parse reads it, serialize writes it – that additionally knows which command carries it, and to_message is the one line that puts the two together.
Three shapes were open when this was written, and the other two are the reason for the one above (issue #1098):
a registry mapping a command to its payload type, so that Message.parse could hand back a typed payload. It is refused, and not narrowly: message.py would import every payload module – tx and block among them once issue #1083’s fourth child lands – so an envelope would cost the whole library to parse, and an unrecognized command would stop being octets and start being an error. The envelope round-tripping what it does not recognize is the property that let it be written before any of this, and a table is what takes it away.
payload classes alone, with the caller writing the command, which is btclib_node’s shape: add_headers(“addr”, payload) is a string literal inside each serialize, and nothing anywhere compares it with the table the receiving side dispatches on. That is how “sendcmpt” and “cmptblock” – both misspelled, both sent to the whole network – survive there. One constant per class, used by both directions, is the whole of the difference.
There is no reverse of `to_message` here, and the asymmetry is the decision rather than an omission. Writing a message needs no table: the payload knows its own command. Reading one back into a typed payload does need a table, so the caller writes it – if message.command == Version.command: Version.parse(message.payload) – and what to do with a command nobody wrote a branch for stays the caller’s, as it is in Core, where net_processing.cpp is a chain of if (msg_type == …) and an unknown type falls off the end. A payload_from_message here would be that table under another name, in a package that would then hold one.
What this costs issue #1083’s remaining children, which inherit it: each of them writes one class per command rather than one entry in a table, and none of them touches message.py. tx and block are thin wrappers over classes that already parse and serialize themselves; addrv2 is a second address class beside the first rather than a flag on it; the inventory and compact-block payloads are ordinary dataclasses. What none of them gets for free is dispatch, which is the line above.
- class btclib.p2p.payload.Payload[source]¶
Bases:
ABCWhat a p2p message carries, and which command carries it.
A subclass is a wire class of this library like any other – a dataclass with parse, serialize and assert_valid – plus command, the message type its octets travel under. to_message is what the pair buys: the command is read off the class instead of being written out at the call, so the name a payload serializes under and the name a caller matches on are one constant.
serialize is declared here and parse is not, which is a fact about the two boundaries rather than an oversight. to_message calls serialize, so the contract has to be stated where it is called; parse is a classmethod every subclass declares for its own return type, and nothing here calls it – this module’s docstring is why there is no from_message to call it from.
An ABC and not a Protocol: to_message is behaviour to inherit rather than a shape to match, and the subclasses are this package’s own. btclib.psbt_signer’s PsbtSigner is the Protocol in this library, and it is one because its implementations are other people’s.
- abstractmethod serialize(*, check_validity: bool = True) bytes[source]¶
Return the wire serialization of the payload alone.
The payload and not the message: what the envelope’s four header fields put in front of it is to_message’s, and a caller holding a Message already has these octets as message.payload.
- to_message(magic: bytes | str | bytearray | memoryview, *, check_validity: bool = True) Message[source]¶
Return this payload framed for a network, ready to send.
magic is the four octets of the message start, which btclib.p2p.magic is where a caller gets and which this library holds no table of; command is the class’s own, so the one thing a caller cannot get wrong here is the name the payload travels under.
btclib.p2p.reject module¶
BIP61’s reject: what a peer refused, and why, and this library parses it.
Sent, per the BIP, in response to a version, a tx or a block that the peer would not accept – and not sent by Bitcoin Core, which removed both directions of it in bitcoin/bitcoin#15437, merged 2019-10-09 and first released in v0.20.0: the pull request’s own words are “parsing of reject messages… completely meaningless” and “the sending of reject messages… a burden”, NetMsgType naming no REJECT since. Core is one peer among the ones that still speak this protocol, and an implementation that has not followed it off the wire sends one all the same – this module is the parser side of BIP37’s own line, issue #1120’s: what a peer sends is parsed, because parsing is not endorsing, and nothing here constructs one to send.
The common payload, and the hash the tx and block cases append to it. message names the command that provoked the reject – “tx”, “block” or “version” – code is BIP61’s one-octet reason, and reason is a human-readable string Core’s own comment already called “for debugging” and never for a caller to act on. A tx or a block reject appends the 32-octet hash of what was refused; a version reject appends nothing, which is the one place a naive parser reads the boundary wrong – the octets that follow reason are either exactly a hash or not there at all, and parse refuses anything between the two.
`message` and `reason` are `str`, decoded as BIP61’s own `var_str` names them rather than held as `user_agent`’s raw octets are. The two are not the same decision for the same reason to differ: handshake.py keeps user_agent undecoded because Core itself sends arbitrary bytes in it today and a refusal here would refuse a message Core accepts. Core sends no reject at all any more, so there is no live sender this parser must not refuse, and message is structurally the same field Message.command already is – a command name, ASCII in every implementation that still emits one, decoded the same way. A peer whose octets are not valid UTF-8 is refused rather than handed back as bytes a caller has to decode with its own error handling.
`code` is `RejectCode` where a member names it and the plain `int` where none does, the same reading btclib.p2p.addrv2._bip155_network_from_int and btclib.p2p.inventory._inventory_type_from_int give a byte-sized code: BIP61 names the codes below and reserves the rest of each range – 0x01-0x0f, 0x10-0x1f, 0x40-0x4f – to “Protocol syntax errors”, “Protocol semantic errors” and “Server policy rule” without naming every member of any of them, so a code no member here has is not malformed, and refusing it would refuse a message BIP61 itself allows.
The refusals that remain are `BTClibValueError` and `BTClibRuntimeError`, the family every payload in this package raises, and never a bare exception from underneath it. A receiver holding this codec in front of a socket sorts a peer’s malformed payload from its own defect by that family – a truncated field, an unrecognized command padding, a hash of the wrong length – exactly as it would for any other message this library parses.
`data` is a `hash256`, held displayed and reversed on the wire, the convention btclib.p2p.inventory.Inventory.hash already carries: what a tx or block reject names is the transaction id or block hash a block explorer prints, not the internal byte order the wire happens to use.
`Reject.parse` takes `Octets` rather than `BinaryData`, for `handshake.Version.parse`’s own reason. The hash is optional and of fixed width, so a no-hash payload for one message, code and reason is a byte-for-byte prefix of the with-hash payload carrying the same three – “where the octets end” is a question about the whole payload, which only the envelope’s length field answers, and a stream holding a reject followed by another message could not tell a present hash from the next message’s first thirty-two octets. Message.parse(…).payload is what a caller already has; tests/parse_contract_test.py names the exclusion this shares with Version.
- class btclib.p2p.reject.Reject(message: str = '', code: int = RejectCode.malformed, reason: str = '', data: bytes | str | bytearray | memoryview = b'', *, check_validity: bool = True)[source]¶
Bases:
PayloadBIP61’s reject message: what a peer refused, and why.
The module docstring has the wire layout, the codec’s own reasons for message and reason being str, and why code round-trips a value BIP61 reserves without naming.
Frozen and hashable, every field being immutable: this is a value, as every payload this package holds is one.
- class btclib.p2p.reject.RejectCode(*values)[source]¶
Bases:
IntEnumBIP61’s named reject codes, Reject.code where a member names it.
Every code the BIP’s tables give a description to, spelled after that description: malformed is the one code common to every message type (“Message could not be decoded”), obsolete and duplicate are version’s own, and the rest answer a tx or a block – invalid covers both of the BIP’s own “is invalid for some reason” rows, one per message type and one number between them. A code this class does not name is not an error: the module docstring is why.
Module contents¶
Module btclib.p2p.
The p2p wire format, and nothing that speaks it. This package turns a message into bytes and bytes back into a message; it is handed the octets and hands octets back, and no line of it opens a socket, resolves a name or waits on one. btclib.fetch is the one place that goes and asks, and its transport is where a socket already is: nothing here imports that package, and nothing there imports this one – a fetcher asks a server a question, a peer is a party to a protocol, and the only thing the two would share is the socket this package refuses to hold.
The envelope, with the payload as opaque bytes. Message is the header Bitcoin Core’s CMessageHeader describes together with the payload it announces, and it reads a command it knows nothing about exactly as it reads one it knows: an envelope that refused an unknown command could not have been written before the payload types, and one that stopped doing so now would refuse the next BIP.
A payload type is a class that knows its own command, and Payload.to_message is what puts one in an envelope; btclib.p2p.payload is where that decision is argued and where what it costs the payload types still to come (issue #1083) is written down. There is no table mapping a command to a type – reading a message back into a typed payload is Version.parse(message.payload) under the caller’s own if, which is the shape net_processing.cpp has too.
The message start is published without being imported. magic_from_chain, magic_from_network and magic_from_signet_challenge are btclib.p2p.magic’s, and that module reaches the bitcoin-core-rpc package’s chains vocabulary, which depends on nothing beyond the standard library – urllib.request, and ssl and socket under it, live in that package’s client and transport instead, which a message-start lookup never reaches. README.md states the property this keeps: “No module loads urllib.request on its way to anything else.” __getattr__ below still answers the three lazily, the same pattern btclib/script/__init__.py uses for sig_hash and engine: import btclib.p2p stays what a parser needs and nothing else, whether or not the module behind a lazy name would itself have been free.
btclib.p2p.limits is not published at all, as btclib.block.limits is not published from btclib.block: a caller reading a protocol constant names the module it comes from, which is what says the number is Core’s and not this library’s.
- class btclib.p2p.Addr(addresses: Sequence[TimestampedNetworkAddress] = (), *, check_validity: bool = True)[source]¶
Bases:
PayloadThe addr message: peers this node knows of, with when it saw them.
A count and that many TimestampedNetworkAddress, which is what Core’s msg_addr writes and what its ProcessMessage reads back.
The count is bounded before anything is built, at Core’s MAX_ADDR_TO_SEND: vAddr.size() > MAX_ADDR_TO_SEND is a Misbehaving there, so a message above it is one no peer accepts. The bound is checked in parse off the count and before the loop, where it is the whole point of having one – the count is the peer’s to choose, and btclib’s own var_int.parse allows 33,554,432 of them, which is the number of thirty-octet objects an implementation without this check builds out of nine octets. That check does not answer to check_validity, on Message.parse’s reasoning: a defence a caller can turn off is not one.
A tuple and not a list, so that a frozen Addr is what its fields say it is: dataclasses.replace is what changes the addresses in one, as it is for every other frozen class here.
- class btclib.p2p.AddrV2(addresses: Sequence[NetworkAddressV2] = (), *, check_validity: bool = True)[source]¶
Bases:
PayloadThe addrv2 message: peers of any network, with when they were seen.
A count and that many NetworkAddressV2, which is Core’s msg_addrv2 and what ProcessMessage reads through the same branch it reads an addr with, CAddress::V2_NETWORK in place of V1_NETWORK being the whole of the difference there.
No base shared with Addr, where Inv and GetData have one: those two are one body under two commands, and these two are two bodies – a TimestampedNetworkAddress and a NetworkAddressV2 are different octets, so what a base could hold is the word “count”.
The count is bounded before anything is built, at MAX_ADDR_TO_SEND, which is BIP155’s thousand and Core’s constant for both commands. As in Addr.parse the check is off the count and before the loop, and it does not answer to check_validity.
A tuple and not a list, so that a frozen AddrV2 is what its field says it is; dataclasses.replace is what changes the addresses in one.
- class btclib.p2p.BIP155Network(*values)[source]¶
Bases:
IntEnumThe network an addrv2 address belongs to, BIP155’s id table.
Every id of that table, spelled as its “Enumeration” column spells them. Core’s own name for the enum, of src/netaddress.h, taken rather than a NetworkId: “network” in btclib already means the chain – btclib.network.Network, and btclib.p2p.magic_from_network beside it – and these are IPv4, Tor and I2P.
TORV2 and YGGDRASIL are members where Core acts on neither, and the module docstring is why. An id no member names is not an error either, which is _bip155_network_from_int.
An IntEnum and not an IntFlag, for the reason InventoryType is one: these are exclusive kinds and not bits, so composing two of them would answer for a network that does not exist.
- class btclib.p2p.BlockFilterType(*values)[source]¶
Bases:
IntEnumWhat a filter type code names, Core’s BlockFilterType.
src/blockfilter.h, and the one code BIP158 defines: “The initial filter types are defined separately in BIP 158”, which defines BASIC and nothing after it.
Core’s enum has a second member, INVALID = 255, and it is not here. That value is what BlockFilter::m_filter_type is initialized to and the one case BuildParams answers false for – a filter that has no type, rather than a type a peer may send – and a library naming it would be publishing a code the protocol has not got. InventoryType leaves out the composite BIP144 reserved for the same reason.
An IntEnum, and _block_filter_type_from_int is what keeps a code no member names from being an error: the module docstring has why an unsupported type is a rule about answering rather than about reading.
- class btclib.p2p.BlockPayload(block: Block, include_witness: bool, *, check_validity: bool = True)[source]¶
Bases:
PayloadThe block message: one block, and the encoding chosen for it.
Bitcoin Core’s msg_block, and TxPayload’s two fields over a Block: a getdata for MSG_BLOCK is answered with every witness stripped and one for MSG_WITNESS_BLOCK with them, which is the same block written twice.
Block.assert_valid is CheckBlock, proof of work included and mainnet’s target by default, and it is what assert_valid here asks of the block: a block message of another network is built with check_validity=False and asked afterwards, which is the same two steps Block.assert_valid already asks of a caller holding one.
Frozen, and not hashable, for the reason TxPayload is not.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) BlockPayload[source]¶
Return the block the payload carries, and how it was written.
Block.is_segwit is any transaction carrying a witness, which is exactly when Block.serialize writes a marker: the flag answers for the message the way each transaction’s marker answers for itself.
- class btclib.p2p.BlockTxn(block_hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', transactions: Sequence[Tx] = (), *, check_validity: bool = True)[source]¶
Bases:
PayloadThe blocktxn message: the transactions a getblocktxn asked for.
BIP152’s BlockTransactions and Core’s class of that name: the block hash and the transactions, “exactly and only each transaction which is present in the appropriate block at the index specified in the getblocktxn indexes list, in the order requested”. block_hash is in display order; the transactions are written with their witnesses, which is BIP152 version 2 and what the module docstring argues.
That the transactions are the ones that were asked for is a property of a connection and not of these octets, so nothing here checks it: what does is PartialBlock.fill, which puts them in the positions the same reconstruction found missing and hands back a Block whose merkle root either commits to them or does not.
Frozen, and not hashable: Tx is a mutable dataclass.
- class btclib.p2p.CFCheckpt(filter_type: int = BlockFilterType.BASIC, stop_hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', filter_headers: Sequence[bytes | str | bytearray | memoryview] = (), *, check_validity: bool = True)[source]¶
Bases:
PayloadThe cfcheckpt message: a filter header every thousand blocks.
Bitcoin Core’s msg_cfcheckpt: the type code, the stop hash, and the vector of filter headers. These are headers and not hashes, so nothing is derived from them – the contrast with cfheaders one class up, which sends the hashes so that the client does the chaining. filter_headers is the field here and the derivation there, which is the one name a caller wants off either message.
heights is what BIP157 says the entries are of: “one entry for each block on the chain terminating in StopHash, where the block height is a multiple of 1,000 greater than 0”.
No count bound, where every other vector in this package has one: BIP157 bounds this one by the length of the chain and Core reads no cfcheckpt at all, so there is no constant to hold it to and none is invented. The module docstring has what stands in front of the loop instead.
Frozen and hashable; dataclasses.replace is what changes the vector.
- property heights: list[int]¶
Return the block height each filter header is that of.
BIP157’s rule read off the vector, CFCHECKPT_INTERVAL being Core’s name for the thousand: the entries are in ascending order by height, so the first is the header of block 1,000 and the last is the highest multiple of a thousand at or below the height of the stop hash.
- class btclib.p2p.CFHeaders(filter_type: int = BlockFilterType.BASIC, stop_hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', previous_filter_header: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', filter_hashes: Sequence[bytes | str | bytearray | memoryview] = (), *, check_validity: bool = True)[source]¶
Bases:
PayloadThe cfheaders message: the filter hashes a header chain is built of.
Bitcoin Core’s msg_cfheaders: the type code, the stop hash, the filter header before the first block of the range, and the vector of filter hashes. Every hash is in display order, BasicBlockFilter.hash’s and BasicBlockFilter.header’s.
The hashes are the field and the headers are derived, which is what BIP157 sends: a client that was handed the headers would have nothing left to check, where the hashes plus one previous header chain into headers it computed itself. filter_headers is that derivation; the module docstring is where storing it instead is refused.
previous_filter_header is thirty-two zero octets for a range starting at the genesis block, which is BIP157’s definition of the header before the first one.
Frozen and hashable, every field being immutable; dataclasses.replace is what changes the vector.
- property filter_headers: tuple[bytes, ...]¶
Return the filter header of each block of the range, in order.
BIP157: a filter header is “the double-SHA256 of the concatenation of the filter hash with the previous filter header”, so the vector plus previous_filter_header is a chain, and the last entry is the header a client compares against what another peer told it. block_filter.filter_header is the one step, the same one BasicBlockFilter.header takes where the filter itself is at hand.
Every hash is in display order, so every header answered is too. A tuple, as CFCheckpt.filter_headers is: one name over the two messages, and one type with it.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) CFHeaders[source]¶
Return the hashes the payload carries, the count bounded first.
BIP157’s “FilterHashesLength MUST NOT be greater than 2,000”, checked before the loop that allocates on it: the count is the peer’s to choose and btclib’s var_int.parse allows 33,554,432 of anything.
- class btclib.p2p.CFilter(filter_type: int = BlockFilterType.BASIC, block_hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', filter_bytes: bytes | str | bytearray | memoryview = b'', *, check_validity: bool = True)[source]¶
Bases:
PayloadThe cfilter message: one block’s filter, and the block it is of.
Bitcoin Core’s msg_cfilter: the type code, the block hash, and the serialized filter behind a CompactSize length. block_hash is in display order, BlockHeader.hash’s.
filter_bytes is BIP157’s FilterBytes – what BasicBlockFilter.serialize writes, the element count and the Golomb-Rice set – and is held as octets: what they encode is the type code’s to say, and only BASIC says anything. basic_filter is the typed reading, and the module docstring is where that is argued and where what this message cannot be checked for is written down.
Frozen and hashable, both octet fields being immutable, which is what holding the filter as octets rather than as a mutable BasicBlockFilter buys.
- assert_valid() None[source]¶
Refuse a type or a block hash the fields cannot hold.
The filter octets are not decoded, whatever the type code says: basic_filter is where they are read as BIP158’s, and refusing them here would make the same octets parse under a type code nobody has defined and fail under the one that is. They round-trip either way, which is the property this package keeps.
Core’s BlockFilter::Unserialize does the opposite and throws “unknown filter_type”, because it builds the Golomb parameters as it reads; deferring that to basic_filter is the whole of the difference, and the module docstring is where it is argued.
Nor are they asked anything else. bytes_from_octets is what __init__ coerced them with and there is no width they must have, so unlike the two hash fields there is nothing left here to refuse – a cfilter of an empty filter is a message BIP158’s own vector file holds.
- property basic_filter: BasicBlockFilter¶
Return the filter these octets are, keyed on this block hash.
The reading and not the field: a cfilter of any other type carries octets no BIP defines, so this is where a caller says the type is BASIC and is refused if it is not. What BasicBlockFilter.parse then refuses is a Golomb stream that does not decode – an element count the bits fall short of, a delta past the range, an octet the deltas never reached.
No argument, which is the seam this message closes: BlockHash precedes FilterBytes in BIP157’s table, so the hash the filter is keyed by arrived with it.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) CFilter[source]¶
Return the filter octets the payload carries, and their block.
NumFilterBytes gets no bound of its own: var_bytes.parse reads the length and then reads from the stream, so what it can build is what the payload holds, and what the payload holds is the envelope’s MAX_PROTOCOL_MESSAGE_LENGTH. A filter has no other limit to be held to – BIP158 bounds the element count and BasicBlockFilter.parse checks that, over octets a caller has already been handed.
- class btclib.p2p.CmpctBlock(header: BlockHeader, nonce: int = 0, short_ids: Sequence[int] = (), prefilled_txns: Sequence[PrefilledTransaction] = (), *, check_validity: bool = True)[source]¶
Bases:
PayloadThe cmpctblock message: a header, short ids, and a few transactions.
BIP152’s HeaderAndShortIDs, which is the whole of the payload – so there is one class here and not a message wrapping a structure, as there is for getblocktxn and blocktxn too. Core’s CBlockHeaderAndShortTxIDs is the same five fields, the two vector lengths being what var_int writes rather than fields of their own.
short_ids are the six-octet integers of the transactions the sender expects the receiver to have, in block order with the prefilled positions taken out; prefilled_txns are the ones it sends whole. Together they are the block: tx_count is their sum, which is BIP152’s “block tx count” read off either vector.
short_id is the derivation the ids come from and short_id_key the key it runs under, both of them functions of the header and the nonce and therefore of this message alone. The module docstring is where the derivation is stated and where what a wrong one would cost is.
Frozen, and not hashable: a PrefilledTransaction holds a mutable Tx. dataclasses.replace is what changes a vector.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) CmpctBlock[source]¶
Return the block this announces, both counts bounded first.
Each vector’s length is read against MAX_BLOCK_TX_INDEX before the loop that allocates on it, btclib’s var_int.parse allowing 33,554,432 of anything; the sum is what assert_valid holds to the same bound afterwards, which is where Core checks it too.
- serialize(*, check_validity: bool = True) bytes[source]¶
Return the header, the nonce, then the two vectors.
- short_id(wtxid: bytes | str | bytearray | memoryview) int[source]¶
Return the six-octet short id this message would carry for a hash.
The hash is a wtxid in BIP152 version 2, Tx.hash, and it is taken in the order this package holds every hash in – the order a block explorer prints – and reversed here, Core hashing the uint256 its own way round. What comes back is the siphash with its two most significant octets dropped, BIP152’s step three.
A hash and not a transaction, which is what leaves version 1 reachable without being offered: the same derivation over Tx.id is the version 1 short id, and which of the two hashes goes in is the negotiated version’s to say rather than this method’s.
- property short_id_key: tuple[int, int]¶
Return the (k0, k1) the short ids of this message are keyed on.
BIP152: single-SHA256 of the header serialization with the nonce appended little-endian, and the first two little-endian 64-bit integers of it. Core’s FillShortTxIDSelector is these lines, and the pair is what btclib.hashes.siphash takes.
Per message and not per block: the nonce is in the digest, so two senders announcing one block under two nonces produce two sets of short ids, which is what BIP152 asks for – “Nodes SHOULD NOT use the same nonce across multiple different blocks” – so that a collision is one peer’s and not the network’s.
- class btclib.p2p.Feature(feature_id: bytes | str | bytearray | memoryview, feature_data: bytes | str | bytearray | memoryview = b'', *, check_validity: bool = True)[source]¶
Bases:
PayloadThe feature message: a feature this peer supports, and its data.
BIP434, and Bitcoin Core’s msg_feature: a featureid and a featuredata, each a length and that many octets. The identifier is what the feature is negotiated under – the BIP number for one published as a BIP, “some other unique identifier” such as a URL or a digest for one that is not – and the data is whatever that feature’s own specification puts there, empty where it wants none.
feature_id has no default, as Message’s command and CmpctBlock’s header have none: an identifier is the whole of what this message says, so an object without one could not be valid.
feature_id is octets and not text, as Version.user_agent is. BIP434 asks for printable ASCII with a SHOULD rather than a MUST, and Core’s p2p_bip434_feature.py asserts a node accepts an identifier that is not – test_non_ascii_feature_id_accepted – so decoding here would refuse a message Core answers. A caller that wants to show one decodes it, with the error handling it wants.
Both lengths are refused rather than parsed, which is the opposite of what FeeFilter does with the money range: BIP434 writes them as a MUST on the encoding and Core answers a payload outside them with a disconnect, where MoneyRange is asked about a value already read. btclib.p2p.limits holds the numbers and the citation.
Frozen and hashable, both fields being immutable.
- class btclib.p2p.FeeFilter(feerate: int = 0, *, check_validity: bool = True)[source]¶
Bases:
PayloadThe feefilter message: do not announce below this fee rate.
BIP133, and Bitcoin Core’s msg_feefilter: eight octets of a fee rate, little-endian, “interpreted as satoshis per kilobyte”, and a peer that sends one is asking not to be told about a transaction paying less. It is a request and not a rule, in BIP133’s own words: the receiving node “will be permitted, but not required, to filter transaction invs for transactions that fall below the feerate provided”. The one thing the BIP says about the filter’s reach is that it does not stop at newly relayed transactions – “Inv’s generated from a mempool message are also subject to a fee filter if it exists”.
The rate is signed, CAmount being int64_t, and a value outside the money range is parsed rather than refused: Core asks MoneyRange before it acts on one, which is what to do with a value rather than whether the octets decode. The module docstring is the whole of that argument.
Frozen and hashable, the one field being immutable.
- class btclib.p2p.GetAddr(*, check_validity: bool = True)[source]¶
Bases:
PayloadThe getaddr message: send me the peers you know of.
Bitcoin Core’s msg_getaddr, and the whole of that command: the answer is an addr, or an addrv2 where the peer asked for one.
Core answers at most once per connection and only an inbound peer – ProcessMessage returns early on pfrom.IsInboundConn() false and on peer->m_getaddr_recvd – and it caps and shuffles what it sends. All three are policy a connection carries out, and none of them is visible in the octets, which is why none of them is here.
- class btclib.p2p.GetBlockTxn(block_hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', indexes: Sequence[int] = (), *, check_validity: bool = True)[source]¶
Bases:
PayloadThe getblocktxn message: which transactions of a block are wanted.
BIP152’s BlockTransactionsRequest and Core’s class of that name: the block hash and the indexes, differentially encoded. block_hash is in display order, BlockHeader.hash’s; indexes are absolute, and the module docstring is where that is argued.
What builds one is PartialBlock.missing_indexes, which is the list a reconstruction that came up short answers with – GetBlockTxn( partial.header.hash, partial.missing_indexes) is the whole of it.
An empty indexes is not refused: Core disconnects the peer that sends one – “No legitimate reason to send indexes empty” – which is a rule about a peer and not about a message, and this package holds none of the first kind.
Frozen and hashable, every field being immutable.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) GetBlockTxn[source]¶
Return the absolute indexes the differences name.
- class btclib.p2p.GetBlocks(version: int = 0, locator: Sequence[bytes | str | bytearray | memoryview] = (), hash_stop: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', *, check_validity: bool = True)[source]¶
Bases:
_LocatorPayloadThe getblocks message: announce the blocks after this point.
Bitcoin Core’s msg_getblocks. The answer is an inv of up to five hundred block hashes, which the asker then fetches with a getdata – one round trip more than getheaders, and what a node without the headers-first sync uses.
- class btclib.p2p.GetCFCheckpt(filter_type: int = BlockFilterType.BASIC, stop_hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', *, check_validity: bool = True)[source]¶
Bases:
PayloadThe getcfcheckpt message: the checkpoints up to a block.
Bitcoin Core’s msg_getcfcheckpt: a type code and a stop hash, and the one request of the three with no start height – a checkpoint chain always begins at the genesis block, so what a client asks for is only where it ends.
A class of its own rather than a third _FilterRangeRequest: two fields are not three, and a start_height here would be a field no message carries.
Frozen and hashable, both fields being immutable.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) GetCFCheckpt[source]¶
Return the chain end the payload asks the checkpoints of.
- class btclib.p2p.GetCFHeaders(filter_type: int = BlockFilterType.BASIC, start_height: int = 0, stop_hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', *, check_validity: bool = True)[source]¶
Bases:
_FilterRangeRequestThe getcfheaders message: the filter hashes of a range of blocks.
Bitcoin Core’s msg_getcfheaders, and the same three fields as getcfilters for a range twice as long: BIP157 bounds this one at “strictly less than 2,000”, limits.MAX_GETCFHEADERS_SIZE, and it is unchecked here for the reason above. The answer is one cfheaders however long the range, the hashes being fixed width.
- class btclib.p2p.GetCFilters(filter_type: int = BlockFilterType.BASIC, start_height: int = 0, stop_hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', *, check_validity: bool = True)[source]¶
Bases:
_FilterRangeRequestThe getcfilters message: the filters of a range of blocks.
Bitcoin Core’s msg_getcfilters. The answer is one cfilter per block, “sequentially in order by block height”, which is the one request here whose answer is many messages.
`limits.MAX_GETCFILTERS_SIZE` is not checked here: BIP157 bounds the range, “the difference MUST be strictly less than 1000”, and the far end of it is a hash. Turning that hash into a height needs the chain, which this package does not hold, so the bound belongs to the caller that does – and the constant is in btclib.p2p.limits under Core’s own name for it.
- class btclib.p2p.GetData(items: Sequence[Inventory] = (), *, check_validity: bool = True)[source]¶
Bases:
_InventoryPayloadThe getdata message: send me these, by identifier.
Bitcoin Core’s msg_getdata, and the one of the three whose codes use the full vocabulary: MSG_FILTERED_BLOCK, MSG_CMPCT_BLOCK and the witness composites “can only occur in getdata”, which is where Inventory.is_witness earns its place.
- class btclib.p2p.GetHeaders(version: int = 0, locator: Sequence[bytes | str | bytearray | memoryview] = (), hash_stop: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', *, check_validity: bool = True)[source]¶
Bases:
_LocatorPayloadThe getheaders message: send the headers after this point.
Bitcoin Core’s msg_getheaders. The answer is a headers message of up to MAX_HEADERS_RESULTS headers, and a shorter one is how the asker learns it has reached the peer’s tip – Core’s comment on that constant says so, and calls changing it a protocol upgrade.
- class btclib.p2p.Headers(headers: Sequence[BlockHeader] = (), *, check_validity: bool = True)[source]¶
Bases:
PayloadThe headers message: block headers, and no transactions.
Bitcoin Core’s msg_headers: a count, and that many eighty-octet headers each followed by a transaction count of zero. The count is not a field of this class and the module docstring is why; serialize writes the zero after every header and parse refuses anything else.
BlockHeader is the type, btclib.block already parsing and serializing one: a header vector is that class in a loop rather than a second reader of the same eighty octets. Which is also what makes the elements answerable to assert_valid_pow – this class does not call it, a headers message being how a node learns of work it has not checked, and the caller is who decides when to.
`MAX_HEADERS_RESULTS` bounds the count before the loop, Core’s own bound on this message and the reason its handler reads the headers by hand: “we don’t want to risk deserializing 2000 full blocks”.
Frozen, and the one class here that is not hashable: BlockHeader is a mutable dataclass, so a tuple of them cannot be hashed. dataclasses.replace is what changes the headers in one.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) Headers[source]¶
Return the headers the payload carries, the count bounded first.
The transaction count after each header is read and refused unless it is zero, where Core reads it and throws it away: a headers carrying any other number is a message this library could not write back, and refusing what it cannot reproduce is what it does everywhere else – Verack.parse on a payload Core ignores, Version.parse on a relay flag Core accepts.
- class btclib.p2p.Inv(items: Sequence[Inventory] = (), *, check_validity: bool = True)[source]¶
Bases:
_InventoryPayloadThe inv message: what this node has, offered unasked.
Bitcoin Core’s msg_inv. An announcement and not a delivery: what a peer does with one is send back a getdata for the entries it wants, which is the next class down.
Core announces transactions as MSG_TX or MSG_WTX and blocks as MSG_BLOCK – “Invs always use TX/WTX or BLOCK”, says the comment in GetDataMsg – and the witness bit belongs to a getdata. That is a rule about what a node sends, not about what these octets can carry, so nothing here refuses the other codes: a peer that sends one is answered by the caller’s own policy, this package holding none.
- class btclib.p2p.Inventory(type_code: int = InventoryType.UNDEFINED, hash: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', *, check_validity: bool = True)[source]¶
Bases:
objectOne entry of an inv, a getdata or a notfound: (type, hash).
Bitcoin Core’s CInv, of src/protocol.h, whose SERIALIZE_METHODS is READWRITE(obj.type, obj.hash): four octets of type code, little-endian, and the thirty-two of a hash256.
type_code is an InventoryType where a member names the code and the plain int where none does, which is how a code this library has not heard of round-trips. Core spells the field type and this library cannot: a class attribute of that name shadows the builtin inside its own body, and type[Inventory] is what every parse here annotates its cls with – “type code” being what the protocol documentation calls it in prose anyway.
hash is a transaction id, a witness transaction id or a block hash depending on that code, and is held in the order a block explorer prints: the order OutPoint.tx_id and BlockHeader.previous_block_hash are held in, these being those very hashes.
Frozen and hashable, both fields being immutable: an entry is a value, and it is what a set of announcements from a peer holds.
- property is_witness: bool¶
Answer whether the code asks for the witness beside the data.
BIP144’s bit, MSG_WITNESS_FLAG, which Core ORs into a type code rather than listing beside it – GetFetchFlags is the one line that sets it, on a peer that may be served witnesses.
The reading and not the field, as Version.is_relay_requested is: comparing type_code against MSG_WITNESS_TX and MSG_WITNESS_BLOCK is the same question asked in a way that goes wrong the day a BIP names a third composite.
- class btclib.p2p.InventoryType(*values)[source]¶
Bases:
IntEnumWhat an inventory entry’s hash identifies, Core’s GetDataMsg.
src/protocol.h, spelled as Core spells it, composites included: MSG_WITNESS_TX and MSG_WITNESS_BLOCK are members there rather than something a caller assembles, so they are members here. The third BIP144 reserved, MSG_FILTERED_BLOCK | MSG_WITNESS_FLAG, is not: Core’s own line for it is commented out as “reserved for future use and remains unused”, and a library naming it would be publishing a code the protocol has not got.
UNDEFINED is Core’s name for zero, which is the code an entry carries when it identifies nothing – CInv’s default, and what its test framework’s type map calls “Error”.
An IntEnum and not an IntFlag: the module docstring has why the ServiceFlags reasoning stops here, and _inventory_type_from_int is what keeps a code no member names from being an error.
GetDataMsg is Core’s name and is not taken here, being the name of one of the three messages that carry the code: an inv announces with it and a notfound refuses with it.
- class btclib.p2p.Mempool(*, check_validity: bool = True)[source]¶
Bases:
PayloadThe mempool message: send me what your mempool holds.
BIP35, and Bitcoin Core’s msg_mempool: the answer is an inv of the transactions in the peer’s mempool, or several, bounded the way any inv is.
Core serves one only where it advertises NODE_BLOOM itself, or where the asking peer holds the mempool permission – the check is on the answering node’s own services and not on the asker’s – and it drops a peer that asks otherwise unless that peer may not be banned. That is policy about who may ask and about what this node offers; the octets of the question are the same either way.
- class btclib.p2p.Message(magic: bytes | str | bytearray | memoryview, command: bytes | str | bytearray | memoryview, payload: bytes | str | bytearray | memoryview = b'', *, check_validity: bool = True)[source]¶
Bases:
objectOne p2p message: which network it is for, what it is, and its payload.
Three fields, where the header has four and the payload follows it, because two of the four are not the object’s to hold. The checksum is the first _CHECKSUM_SIZE octets of hash256(payload) and is a property, so that no instance can carry one disagreeing with the payload beside it; the payload length is len(payload) for the same reason. What is left – the magic, the command and the payload – is what the octets do not determine.
magic is four octets and is looked up nowhere: an unfamiliar one round-trips, a custom signet’s being derived from its challenge rather than tabulated – btclib.p2p.magic is where a caller gets one and where that decision is argued.
It is a field of the message and not something a connection puts in front of it, which is where this departs from btclib_node: there messages.add_headers writes the other three fields and p2p.connection.Connection._send prepends the magic, so what the codec serializes is not a message and is not what its own verify_headers reads back – that one indexes the length at 16, which is where it sits once the magic is there. One header across two layers is also the one shape unavailable to a package that holds no connection.
command is the message type as text, “version” or “verack”, without the NUL padding the wire puts after it, and an unknown one round-trips as opaque bytes do: this class is the envelope, and it knows no payload type (issue #1083).
Text and not the twelve octets verbatim, which is the choice a round-trip is usually the argument against: strip the padding and two wire values decode to one object, which is the malleability assert_no_trailing is spent on one field down. What answers it is not keeping the octets but refusing the ones Core refuses – _command_from_bytes is IsMessageTypeValid – so the twelve octets and the text are one to one in both directions, and every value this accepts serializes back to the value it was read from. Keeping the field verbatim would do the opposite of what it looks like: it would round-trip a “ping” with a stray octet after its NUL faithfully, and that is a header Core drops the sender for, so btclib would be reading and re-emitting a message no peer accepts. It would also put the padding in every caller’s hands, where String and str_from_string are what the rest of this library uses for a field that is ascii text.
Frozen, all three fields being immutable: a message is a value, and dataclasses.replace is what retargets one at another network.
No to_dict and no from_dict, where the other wire-format classes of this library have both: those agree with a json shape somebody else writes too – Core’s rpc for a transaction, BIP174’s for a psbt – and nothing renders a p2p envelope as json, so the pair would be inventing a shape rather than reading one, over a payload that is opaque octets either way.
- assert_valid() None[source]¶
Refuse a magic, a command or a payload no message carries.
The payload bound is MAX_PROTOCOL_MESSAGE_LENGTH: a message above it is one no peer accepts, so serializing one would be writing octets with nowhere to go. parse refuses the same bound and it is not the same check – there it is read off the length field before the payload is allocated, and it cannot be turned off.
- property checksum: bytes¶
Return the four octets the header carries.
The first four octets of hash256(payload), Bitcoin Core’s V1Transport::GetMessageHash. It is what the payload says it is, so it is derived here and never stored; parse is the one place the two can disagree, and it refuses the octets rather than building the disagreement.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) Message[source]¶
Return one message, telling “not all there yet” from “never”.
A BytesIO is the caller’s stream and may hold part of the next message, or several whole ones: what is consumed is this message, and the stream is left on the octet after it, which is how a caller reading off a socket takes them one at a time – the position being what says how much was consumed. Octets are one whole object and what follows the message in them is refused, as everywhere in this library; btclib.utils states both halves.
Octets that end inside the message raise `IncompleteMessageError`, and the stream is rewound to where the message started. It is the one refusal here that more octets can answer, so the caller reads missing more, appends them and calls again on a stream still positioned at the start. Every other refusal is final and none of them rewinds: a peer whose header does not decode is a peer Core disconnects rather than resynchronizes with, and what to do about it is the caller’s policy, not this package’s.
The header is read as one unit rather than field by field, which is Bitcoin Core’s own split – V1Transport has a header phase and a body phase – and is what makes missing exact: the octets still wanted are the rest of the header, and once the header is in hand, the rest of the payload. A field-by-field read can only report what the field it stopped in was short of, which is not a number the caller can ask its socket for.
The payload length is compared with MAX_PROTOCOL_MESSAGE_LENGTH before the payload is asked for: the field is the peer’s to choose, and the whole of what the bound is for is that nothing allocates on it first. That check does not answer to check_validity – a defence a caller can turn off is not one – and neither does the checksum, which is what tells this payload from the octets a link corrupted: skipped, two buffers would decode to the one object that serializes back to only one of them.
The magic is read and not checked, no argument here naming the network expected. A caller that means to refuse another chain’s message compares message.magic with what it expects, which is the one line an optional magic=None here would replace with a defence that is off unless asked for; and a magic no table holds is a custom signet’s, which must round-trip rather than be refused.
- class btclib.p2p.NetworkAddress(services: int = <ServiceFlags.NODE_NONE: 0>, ip: IPv4Address | IPv6Address | str | bytes = '::', port: int = 0, *, check_validity: bool = True)[source]¶
Bases:
objectWhere a peer is and what it offers: (services, ip, port).
Bitcoin Core’s CService with the service flags in front of it, in the twenty-six octets a CAddress writes under Encoding::V1 with no timestamp in front – which is what a version message’s addr_recv and addr_from are. TimestampedNetworkAddress is the thirty-octet form an addr message carries, and the module docstring is why they are two classes.
ip is an ipaddress.IPv6Address and is always sixteen octets, an IPv4 peer being ::ffff:a.b.c.d: ip.ipv4_mapped is the v4 address where there is one and None where there is not, which is the question a caller would otherwise keep a tag for. The constructor takes any of the spellings IPAddress names, so “10.0.0.1” and “::ffff:10.0.0.1” build the one object – as they must, being the one peer.
services is a ServiceFlags, which is an int carrying the bits it cannot name; port is the one big-endian field in this protocol.
Frozen and hashable, all three fields being immutable: an address is a value, it is what an address database keys on, and dataclasses.replace is what moves one to another port.
No to_dict and no from_dict, for the reason Message has none: those agree with a json shape somebody else writes, and the shape Core’s rpc renders a peer as – getpeerinfo’s “addr” – is a formatted string and this structure’s fields spread across a dozen other keys, so the pair would be inventing one rather than reading one.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) NetworkAddress[source]¶
Return the address the twenty-six octets describe.
- class btclib.p2p.NetworkAddressV2(timestamp: int = 0, services: int = <ServiceFlags.NODE_NONE: 0>, network_id: int = BIP155Network.IPV4, address: bytes | str | bytearray | memoryview = b'\x00\x00\x00\x00', port: int = 0, *, check_validity: bool = True)[source]¶
Bases:
objectOne entry of an addrv2: a peer, and the network it is on.
Bitcoin Core’s CAddress written with CAddress::V2_NETWORK, which is BIP155’s table of fields in its order: timestamp in four octets little-endian, services as a CompactSize, network_id in one octet, address as var_bytes, and port in two octets big-endian.
address is the octets and nothing more – the network id says how to read them, and the module docstring is why nothing here does. network_id is a BIP155Network where a member names the id and the plain int where none does, which is how an address of a network this library has not heard of comes back as it arrived. services is a ServiceFlags, which is an int carrying the bits it cannot name.
A port of zero is what BIP155 requires where a port means nothing for the network, and is a value like any other here.
Frozen and hashable, every field being immutable: an address is a value, it is what an address database keys on, and dataclasses.replace is what moves one to another port.
- assert_valid() None[source]¶
Refuse a field no width holds, and an address of the wrong length.
The length is BIP155’s table read against network_id: an id a member names fixes it, and a mismatch is what the BIP calls meaningless and what Core’s SetNetFromBIP155Network throws on. An id no member names fixes nothing, so MAX_ADDRV2_SIZE is the whole of what such an address is held to.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) NetworkAddressV2[source]¶
Return the entry the octets describe, the address bounded first.
MAX_ADDRV2_SIZE is checked off the length field and before the read it would size, which is where a bound is worth having: the length is the peer’s to choose. It does not answer to check_validity, on Message.parse’s reasoning – a defence a caller can turn off is not one – where the table check in assert_valid does, being a statement about the address rather than about what reading it costs.
- class btclib.p2p.NotFound(items: Sequence[Inventory] = (), *, check_validity: bool = True)[source]¶
Bases:
_InventoryPayloadThe notfound message: I have none of these.
Bitcoin Core’s msg_notfound, the answer to a getdata naming something this node cannot serve – a transaction that has left its mempool, most often.
`MAX_INV_SZ` bounds this one too, where Core does not bound it, and the difference is worth stating rather than glossing: Core refuses an over-long inv or getdata with a Misbehaving and answers an over-long notfound by ignoring its contents instead. A parser has to bound it all the same – a count is the peer’s to choose, and this is the same loop the other two run – and MAX_INV_SZ is the right number for it because a notfound answers a getdata, which cannot have held more.
- class btclib.p2p.PartialBlock(header: BlockHeader, transactions: Sequence[Tx | None] = (), *, check_validity: bool = True)[source]¶
Bases:
objectA block with the transactions found so far in it, and gaps for the rest.
What reconstruct answers with, and Core’s PartiallyDownloadedBlock without the word this package cannot use: the header, and one entry per transaction of the block, each either a transaction or None. missing_indexes is what is still wanted and fill is what finishes the block once it has arrived.
Frozen, and not hashable: the entries are mutable Tx objects.
- fill(transactions: Sequence[Tx] = (), *, check_validity: bool = True) Block[source]¶
Return the block, the gaps filled with the transactions supplied.
transactions are a blocktxn’s, in the order missing_indexes asked for them, and there must be exactly as many: a shorter answer leaves a position empty and a longer one names a position nothing asked about, which is what Core’s FillBlock refuses on both sides of its loop.
What is not checked here is that they are the right transactions, and the block is where that shows: a short id collision that survived the pool puts a wrong transaction in a position, and the merkle root Block.assert_valid recomputes is what does not then commit to it – Core reaches for the same answer, calling IsBlockMutated at the end of FillBlock and calling what it catches “Possible Short ID collision”.
check_validity is passed to Block, whose assert_valid is Core’s CheckBlock with mainnet’s target: a block of another network is built with it cleared and asked afterwards, which is the two steps btclib.p2p.data’s BlockPayload asks of a caller too.
- property missing_indexes: list[int]¶
Return the indexes of the transactions still wanted, in order.
Exactly what a getblocktxn names, and in the order a blocktxn answers in: GetBlockTxn(partial.header.hash, partial.missing_indexes) is the request, and fill takes what comes back.
A list rather than an exception, which is the whole of the decision reconstruct owes: a reconstruction that came up short has an answer worth having, and it is this one.
- class btclib.p2p.Payload[source]¶
Bases:
ABCWhat a p2p message carries, and which command carries it.
A subclass is a wire class of this library like any other – a dataclass with parse, serialize and assert_valid – plus command, the message type its octets travel under. to_message is what the pair buys: the command is read off the class instead of being written out at the call, so the name a payload serializes under and the name a caller matches on are one constant.
serialize is declared here and parse is not, which is a fact about the two boundaries rather than an oversight. to_message calls serialize, so the contract has to be stated where it is called; parse is a classmethod every subclass declares for its own return type, and nothing here calls it – this module’s docstring is why there is no from_message to call it from.
An ABC and not a Protocol: to_message is behaviour to inherit rather than a shape to match, and the subclasses are this package’s own. btclib.psbt_signer’s PsbtSigner is the Protocol in this library, and it is one because its implementations are other people’s.
- abstractmethod serialize(*, check_validity: bool = True) bytes[source]¶
Return the wire serialization of the payload alone.
The payload and not the message: what the envelope’s four header fields put in front of it is to_message’s, and a caller holding a Message already has these octets as message.payload.
- to_message(magic: bytes | str | bytearray | memoryview, *, check_validity: bool = True) Message[source]¶
Return this payload framed for a network, ready to send.
magic is the four octets of the message start, which btclib.p2p.magic is where a caller gets and which this library holds no table of; command is the class’s own, so the one thing a caller cannot get wrong here is the name the payload travels under.
- class btclib.p2p.Ping(nonce: int = 0, *, check_validity: bool = True)[source]¶
Bases:
_NoncePayloadThe ping message: a nonce a pong is expected to echo back.
A nonce of zero is a nonce, which is worth saying because it is the one an implementation reading its own field for truth invents a replacement for – and Core does send zero, ProcessMessage declaring uint64_t nonce = 0 and writing back whatever it read. There is no default nonce here beyond the field’s own: choosing one is drawing a random number, which is the caller’s and secrets’.
- class btclib.p2p.Pong(nonce: int = 0, *, check_validity: bool = True)[source]¶
Bases:
_NoncePayloadThe pong message: the nonce of the ping it answers.
Whether it is the nonce that was sent is the caller’s question and not this codec’s: matching an answer to a question is what a connection does, and this package holds no connection.
- class btclib.p2p.PrefilledTransaction(index: int, tx: Tx, *, check_validity: bool = True)[source]¶
Bases:
objectOne transaction a cmpctblock carries whole, and where it belongs.
BIP152’s PrefilledTransaction and Core’s struct of that name: an index and the transaction at it. The sender puts here what it expects the receiver has not got – always the coinbase, which is in no mempool, and “a select few which we expect a peer may be missing”.
index is the absolute index into the block, which is BIP152’s own description of the field; the wire carries the difference from the previous one, minus one, and previous_index is what serialize and parse take that difference against. _NO_PREVIOUS_INDEX is its default and is what makes a standalone one the first of a list: a written zero is index zero. The module docstring is where holding the absolute index rather than the wire’s own is argued.
Not a Payload: no command carries a prefilled transaction on its own, cmpctblock being the message and this a structure inside it.
Frozen, and not hashable: Tx is a mutable dataclass, so the field cannot be hashed and dataclasses.replace is what moves one.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, previous_index: int = -1, *, check_validity: bool = True) PrefilledTransaction[source]¶
Return the transaction and the index the difference names.
The difference is bounded before it is added, and the sum after: both are MAX_BLOCK_TX_INDEX, which is what Core’s DifferenceFormatter::Unser refuses a running index past.
- serialize(previous_index: int = -1, *, check_validity: bool = True) bytes[source]¶
Return the difference from the previous index, then the transaction.
The transaction is written with its witness, BIP152 version 2’s “same format as responses to getdata MSG_WITNESS_TX” – which for a transaction that has no witness is the same octets version 1 would have written, the marker going in only where there is something to mark.
- class btclib.p2p.Reject(message: str = '', code: int = RejectCode.malformed, reason: str = '', data: bytes | str | bytearray | memoryview = b'', *, check_validity: bool = True)[source]¶
Bases:
PayloadBIP61’s reject message: what a peer refused, and why.
The module docstring has the wire layout, the codec’s own reasons for message and reason being str, and why code round-trips a value BIP61 reserves without naming.
Frozen and hashable, every field being immutable: this is a value, as every payload this package holds is one.
- class btclib.p2p.RejectCode(*values)[source]¶
Bases:
IntEnumBIP61’s named reject codes, Reject.code where a member names it.
Every code the BIP’s tables give a description to, spelled after that description: malformed is the one code common to every message type (“Message could not be decoded”), obsolete and duplicate are version’s own, and the rest answer a tx or a block – invalid covers both of the BIP’s own “is invalid for some reason” rows, one per message type and one number between them. A code this class does not name is not an error: the module docstring is why.
- class btclib.p2p.SendAddrV2(*, check_validity: bool = True)[source]¶
Bases:
PayloadThe sendaddrv2 message: no fields, and an empty payload.
Core’s msg_sendaddrv2, and the whole of that command: a peer that sends one is saying it understands addrv2 and would rather have it than addr.
BIP155 puts it in the handshake – it “MUST only be sent in response to the version message from a peer and prior to sending the verack message”, and Core disconnects a peer that sends one after the verack. That is a rule about when, which needs a connection to hold; this package has none, so the rule is documented here and nothing enforces it. The message lives beside addrv2 rather than in btclib.p2p.handshake because it is about nothing else.
parse refuses an octet, as Verack.parse does and for that class’s reason: this library refuses what follows an object everywhere else, and a sendaddrv2 with a payload is a message that serializes back without it. The two classes repeat three lines rather than share a base – keepalive._NoncePayload is a base because two commands have one body, and the absence of a body is not one to share.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) SendAddrV2[source]¶
Return a SendAddrV2, refusing any octet at all.
Octets are refused by assert_no_trailing, which is where the rule already is; a caller’s stream is left exactly where it was, a sendaddrv2 consuming nothing from one.
- class btclib.p2p.SendCmpct(announce: bool = False, version: int = 2, *, check_validity: bool = True)[source]¶
Bases:
PayloadThe sendcmpct message: whether to announce, and in which version.
Bitcoin Core’s msg_sendcmpct: one octet read as a boolean and eight of version, little-endian. announce set is BIP152’s high-bandwidth mode, “the node SHOULD announce new blocks by sending a cmpctblock message”; cleared is the low-bandwidth mode, where blocks are announced with inv or headers and a compact one is asked for.
version defaults to CMPCTBLOCKS_VERSION, which is the encoding this module implements and the only one Core answers to – and the field is written and read unchanged whatever it says, that being what it is for: BIP152 negotiates by each side naming the versions it will speak, so a message naming a version this library does not is a message it must still be able to read.
Frozen and hashable, both fields being immutable.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) SendCmpct[source]¶
Return what the peer announced, the first octet being one or zero.
BIP152: the first integer “SHALL be interpreted as a boolean (and MUST have a value of either 1 or 0)”, so anything else is refused rather than read as true. Core reads the octet through its own bool deserialization, which takes any non-zero value and cannot write back what it read either; refusing is the reading that keeps the octets a caller sends the octets that arrived.
- class btclib.p2p.SendHeaders(*, check_validity: bool = True)[source]¶
Bases:
PayloadThe sendheaders message: announce a new block as a header.
BIP130, and Bitcoin Core’s msg_sendheaders: a peer that sends one is asking to be told about a new block with a headers message rather than with an inv it would then have to ask about. The saving is the round trip, which is BIP130’s own argument.
There is no message that turns it off again, and no rule that it be honoured either. BIP130 is permissive in both directions: the receiving node “will be permitted, but not required, to announce new blocks by sending the header”, and implementations “may also optionally impose additional constraints, such as only honoring sendheaders messages shortly after a connection is established”. Core imposes none of them – m_prefers_headers = true is the only write there is, and it happens whenever the message arrives.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) SendHeaders[source]¶
Return a SendHeaders, refusing any octet at all.
- class btclib.p2p.SendTxRcncl(version: int = 0, salt: int = 0, *, check_validity: bool = True)[source]¶
Bases:
PayloadThe sendtxrcncl message: this peer would reconcile, not announce.
BIP330, and Bitcoin Core’s msg_sendtxrcncl: a uint32 protocol version and a uint64 salt, this peer’s half of the entropy the two sides combine – TaggedHash(“Tx Relay Salting”, salt1, salt2), the lower salt first – to key the short transaction IDs a reconciliation round exchanges. The module docstring has the whole of what this message is the negotiation for and is not the codec of.
version is 1 for every peer running the protocol BIP330 and node/txreconciliation.h describe today, and this class does not enforce that: a version below 1 is what Core’s TxReconciliationTracker::RegisterPeer calls a protocol violation, after comparing the two peers’ versions against each other, which this codec never sees. salt is entropy, and every value its width holds is one a peer may have chosen.
Frozen and hashable, both fields being immutable.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) SendTxRcncl[source]¶
Return the version and the salt the twelve octets carry.
- class btclib.p2p.ServiceFlags(*values)[source]¶
Bases:
IntFlagThe services a node advertises, Bitcoin Core’s ServiceFlags.
src/protocol.h, spelled as Core spells it, and a bit set rather than a table: the eight octets are a bitfield, an unknown bit is a service this library has not heard of rather than an error, and Core says so where it reserves bits 24-31 “for temporary experiments” and sends everything else through the BIP process.
An IntFlag is what round-trips such a bit: a value with bits no member names keeps them and compares equal to the integer it was built from, so ServiceFlags(1 << 40) serializes back to the octets it was parsed from. The members are what a caller reads the named bits with – ServiceFlags.NODE_WITNESS in flags – and neither parse nor assert_valid consults them.
Bit 1 is absent because Core removed it: it was BIP64’s NODE_GETUTXO, and a version still carrying it is exactly the unnamed bit above. serviceFlagsToStr is Core’s own answer to the same question, and it answers “UNKNOWN[…]” rather than refusing.
- class btclib.p2p.TimestampedNetworkAddress(timestamp: int = 0, address: NetworkAddress | None = None, *, check_validity: bool = True)[source]¶
Bases:
objectOne entry of an addr message: when a peer was last seen, and where.
The thirty octets Bitcoin Core writes for a CAddress on the network – four of nTime, then the twenty-six a NetworkAddress is. The timestamp is unsigned and four octets wide where a version message’s is signed and eight, which is the second reason these are two structures rather than one with a flag: the same name in Core’s prose is not the same field.
Composed rather than inherited, so that a Version cannot be handed one: a subclass of NetworkAddress would satisfy that annotation and serialize four octets nobody asked for, which is the trap this module’s docstring names in the implementation that has it.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) TimestampedNetworkAddress[source]¶
Return the entry the thirty octets describe.
- class btclib.p2p.TxPayload(tx: Tx, include_witness: bool, *, check_validity: bool = True)[source]¶
Bases:
PayloadThe tx message: one transaction, and the encoding chosen for it.
Bitcoin Core’s msg_tx. tx is the transaction and include_witness is what Tx.serialize takes – BIP144’s question, answered by the connection and held here so that Payload.serialize keeps one signature; the module docstring is where that is argued and where what parse can and cannot recover is written down.
include_witness has no default, where Block.serialize’s has one: a message is written for a peer, and which encoding that peer negotiated is not a value this package can pick on its behalf.
Frozen, and not hashable: Tx is a mutable dataclass, so the field cannot be hashed and dataclasses.replace is what re-encodes a payload for another peer.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) TxPayload[source]¶
Return the transaction the payload carries, and how it was written.
Tx.parse reads BIP144’s marker, so is_segwit is what the octets said; the module docstring has what that answer cannot distinguish and why nothing here reads the marker a second time.
- class btclib.p2p.Verack(*, check_validity: bool = True)[source]¶
Bases:
PayloadThe verack message: no fields, and an empty payload.
Bitcoin Core’s msg_verack, which serializes to nothing and whose deserialize reads nothing.
A class all the same, and the empty payload is why rather than despite: it is the one payload type whose whole content is its command, so the constant on it is the entire benefit of having a class at all, and it is what proves the shape btclib.p2p.payload.Payload states is uniform. Verack().to_message( magic) is a complete verack, resting on the envelope’s payload=b”” default.
parse refuses an octet, where Core ignores whatever a verack carries – ProcessMessage never reads vRecv for one. This library refuses what follows an object everywhere else, and a verack with a payload is a message that serializes back without it.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) Verack[source]¶
Return a Verack, refusing any octet at all.
Octets are refused by assert_no_trailing, which is where the rule already is; a caller’s stream is left exactly where it was, a verack consuming nothing from one.
- class btclib.p2p.Version(version: int = 70016, services: int = <ServiceFlags.NODE_NONE: 0>, timestamp: int = 0, addr_recv: NetworkAddress | None = None, addr_from: NetworkAddress | None = None, nonce: int = 0, user_agent: bytes | str | bytearray | memoryview = b'', start_height: int = 0, relay: bool | None = None, *, check_validity: bool = True)[source]¶
Bases:
PayloadThe version message: who is calling, and what it can do.
Nine fields, of which the last may not be on the wire at all. In the order Core serializes them: the protocol version, the service flags, the sender’s clock, the address it is writing to, the address it is writing from, a nonce it recognizes its own connection by, the user agent, the height of its best chain, and BIP37’s relay flag.
relay is True, False or None, and None is not False: it says the octets ended before the flag, which is what a peer older than BIP37 sends and what Core still accepts from any peer. What such a peer means is True – net_processing.cpp initializes bool fRelay = true and overwrites it only where the field is there – and is_relay_requested is that reading, so that a caller does not write if version.relay and answer the opposite of the protocol’s default for every peer that omitted it. The module docstring is why the presence is a field rather than a function of version.
user_agent is octets and not text. Core reads it into a std::string and sanitizes it only for the log – SanitizeString on cleanSubVer – so a peer may put anything at all in it, and decoding here would refuse a message Core accepts. A caller that wants to show one decodes it, with the error handling it wants. MAX_SUBVERSION_LENGTH is the bound on it, Core’s LIMITED_STRING.
timestamp and start_height are signed, version is signed, and services and nonce are not, each following the type Core declares – a negative nTime is what Core clamps to zero on receipt rather than refusing, so it is a value this parses.
addr_recv and addr_from carry no timestamp, which is btclib.p2p.address’s reason for two classes.
version defaults to limits.PROTOCOL_VERSION, Core’s own number and the one a Version built with no argument for it announces – not because this library speaks for a peer’s protocol version, which parse never does, but because a caller building one to send is building this library’s own handshake, and PROTOCOL_VERSION is what that is.
- property is_relay_requested: bool¶
Answer whether this peer wants transactions announced to it.
BIP37’s flag, with BIP37’s default where the flag is absent: Bitcoin Core’s net_processing.cpp declares bool fRelay = true before it reads the message and assigns to it only inside if (!vRecv.empty()), so a version that stops before the flag asks for relay rather than refusing it.
The reading and not the field, which is what relay is: a caller that needs to know whether the peer said so reads relay is None. Reading relay itself for the answer is what makes an absent flag mean the opposite of what the protocol says it means.
- classmethod parse(data: bytes | str | bytearray | memoryview, *, check_validity: bool = True) Version[source]¶
Return the version the payload describes, relay flag or not.
The flag is read where an octet is left and left None where none is, which is the one conditional here; everything before it is required, and the module docstring is why.
Octets and not BinaryData, which is the other half of that decision: “where an octet is left” is a question about the whole payload, and in a stream holding the next message the answer would be the first octet of that one. The envelope is what says where a payload ends, so message.payload is what this takes.
Only 0x00 and 0x01 are a flag. Core’s Unserialize for a bool takes any octet and answers != 0, so 0x02 reads as true there and is written back as 0x01 – two payloads, one object, and only one of them serialized back. That is the malleability Message’s command padding is refused for one layer down, and the same answer is given here.
- class btclib.p2p.WtxidRelay(*, check_validity: bool = True)[source]¶
Bases:
PayloadThe wtxidrelay message: announce transactions by wtxid.
BIP339, and Bitcoin Core’s msg_wtxidrelay: a peer that sends one is asking for MSG_WTX in the inventories it is sent, so that a transaction is named by the hash that commits to its witness and a witness-malleated copy is a different announcement rather than the same one.
One message enables one direction. BIP339: “After a node has received a wtxidrelay message from a peer, the node MUST use the MSG_WTX inv type when announcing transactions to that peer” – so a peer that sends one has said how it wants to be announced to, and said nothing about how it will announce. Core matches, setting m_wtxid_relay on receipt alone and never consulting whether it sent its own; two Core nodes both send one, which makes the connection symmetric in practice and is a fact about Core rather than about the message.
Like BIP155’s sendaddrv2, it belongs between the version and the verack, and Core disconnects a peer that sends one after. That is a rule about when, and holding it needs a connection this package does not have.
- classmethod parse(data: BytesIO | bytes | str | bytearray | memoryview, *, check_validity: bool = True) WtxidRelay[source]¶
Return a WtxidRelay, refusing any octet at all.
- btclib.p2p.addr_entry(address: NetworkAddressV2) TimestampedNetworkAddress[source]¶
Return the addr entry a BIP155 record can_addrv1 allows.
- btclib.p2p.can_addrv1(address: NetworkAddressV2) bool[source]¶
Answer whether an addr message has room for this peer.
Both IP networks and neither of the others: the question is about the network being an IP one at all, not about whether the address is otherwise worth dialling, which is a node’s own policy and not this package’s.
- btclib.p2p.is_embedded_ipv6(address: NetworkAddressV2) bool[source]¶
Answer whether an IPV6 record’s octets are really another network’s.
BIP155’s two ignore rules together: a v4-mapped address, and one inside OnionCat’s range, once used to carry a TORv2 address the same way. Both are receive policy, about whether an address is worth keeping rather than whether the octets decode – the module docstring is why assert_valid does not apply this – so it is a caller deciding what to keep that does.
- btclib.p2p.magic_from_chain(chain: str) bytes[source]¶
Return the p2p message start of one of Core’s chains.
Signet’s is the default signet’s – magic_from_signet_challenge is what answers for any other, the challenge being what a signet is identified by.
- btclib.p2p.magic_from_network(network: str = 'mainnet') bytes[source]¶
Return the p2p message start of one of btclib’s networks.
The four octets every message on that network begins with, from Bitcoin Core’s pchMessageStart per chain. network is a NETWORKS name in any case and spaced how it likes, as everywhere a network is named; signet’s answer is the default signet’s, another signet being identified by its challenge rather than by a name, and magic_from_signet_challenge is what answers for that one.
A name no network has leaves as a BTClibValueError and a value that is no name at all as a BTClibTypeError, both from the converter this shares with the rest of the library rather than from the package the table is in.
- btclib.p2p.magic_from_signet_challenge(challenge: str | bytes | bytearray) bytes[source]¶
Return the p2p message start a signet’s block challenge determines.
Core: “message start is defined as the first 4 bytes of the sha256d of the block script”, the script serialized with its CompactSize length, and the four bytes in the order the digest produces them. BIP325 is the challenge itself; this is what SigNetParams does with it.
Hex or the bytes it spells, because a challenge is written in a config file and reported by getblockchaininfo as hex, and held as bytes by anything that has parsed it. Nothing else, bytes(7) being seven zero bytes rather than an error: a challenge that arrived as a number would otherwise be hashed as a script of that length.
A challenge no node would accept is refused: Core takes at least one byte, and above two bytes of length prefix the serialization is one this does not write.
- btclib.p2p.network_address(address: NetworkAddressV2) NetworkAddress[source]¶
Return the untimestamped form of a BIP155 record can_addrv1 allows.
What a version message’s two addresses are, and what an addr entry is built on. can_addrv1 is the question a caller asks first; the refusal here is what makes the answer binding rather than advisory, because the length would not catch it on its own: BIP155 gives cjdns and yggdrasil the sixteen octets an IPv6 address has, so IPv6Address would take either for an IP address and hand back a peer that is not the one that was gossiped.
- btclib.p2p.peer_from_addr_entry(entry: TimestampedNetworkAddress) NetworkAddressV2[source]¶
Return the BIP155 record an addr entry describes.
An addr entry holds every address in sixteen octets, a v4 one mapped into them, where BIP155 gives the two networks different ids and different lengths: ip.ipv4_mapped is what tells them apart, and it is why this is not a field rename.
- btclib.p2p.reconstruct(compact_block: CmpctBlock, pool: Sequence[Tx] = ()) PartialBlock[source]¶
Return the block a compact one and a pool of transactions make.
The prefilled transactions go in the positions they name, and every short id is looked for among the pool; what is left over is PartialBlock.missing_indexes, which is what a getblocktxn asks for. Core’s PartiallyDownloadedBlock::InitData is the same walk, over a mempool and an extra pool rather than over one sequence: which transactions are candidates is the caller’s to decide, this package holding no mempool.
The pool is matched by wtxid, Tx.hash, which is BIP152 version 2’s short id input; the module docstring has why version 1 is not offered and what it would change.
Two refusals, and they are BIP152’s rather than this library’s:
a compact block of no transactions is no block, there being no block without a coinbase. Core’s InitData answers READ_STATUS_INVALID for it;
a compact block whose own short ids are not unique cannot be reconstructed, two positions wanting one transaction, and BIP152’s answer is to ask for the block the ordinary way. Core’s is READ_STATUS_FAILED with “Short ID collision” beside it. It is refused here and not in CmpctBlock.assert_valid, such a message being one a peer legitimately sends.
A pool collision is the third case and is not a refusal: where two different transactions of the pool answer one short id, the position is left missing and is asked for, which is what Core does and why – “eating a round-trip due to FillBlock failure would be annoying”. Taking the first match instead is the bug this shape exists to refuse.
The arguments are checked before any of that, which a free function taking an object a caller already built has to do for itself: a CmpctBlock is what the first has to be and a sequence of Tx the second, so that “not a compact block at all” leaves as this library’s own exception rather than as an AttributeError about a field name. psbt.assert_signatures_only is the precedent, and tests/built_object_contract_test.py the gate over the family.
What is not re-asked is CmpctBlock.assert_valid: a message built with check_validity=False is the caller’s own here as everywhere else in this library. Two of its checks are asked all the same, being what the walk below rests on – _assert_positions, a prefilled index outside the block being an IndexError and not an answer, and the header, which is read for the short id key.
There is no check_validity of its own, and that is because there would be nothing left for it to turn off: this is not a constructor a caller hands fields to, so everything the PartialBlock holds either arrived inside the message or was checked on the way in, and the PartialBlock is therefore built with the flag cleared.