Performance Benchmarks

This document presents performance benchmarks for genro-bag operations, comparing them against Python’s built-in dict where applicable.

Running the Benchmarks

python benchmarks/benchmark_bag.py

Test Environment

Results may vary based on hardware. The benchmarks were run on a typical development machine with Python 3.12.

Summary of Results

Operation

Performance

Notes

Empty Bag creation

~0.23 µs

~10x slower than dict

Direct key access

~0.6 µs

Comparable to dict

Nested path access

~1.2 µs

5-level deep path

Simple assignment

~2 µs

Includes node creation

Iteration

~30 µs/1k nodes

Very efficient

XML serialization

~0.18 ms/100 nodes

TYTX JSON

~0.09 ms/100 nodes

2x faster than XML

TYTX MsgPack

~0.06 ms/100 nodes

3x faster than XML

Memory Consumption

Understanding memory usage is important for applications handling large datasets.

Basic Memory Overhead

Empty dict: 64 bytes
Empty Bag: 48 bytes (shallow)

An empty Bag is actually smaller than an empty dict in shallow size, but the real cost comes from the BagNode objects created for each item.

Scaling with Size

Items

dict

Bag

Ratio

Per-item overhead

100

7.7 KB

36 KB

4.7x

~290 bytes

10,000

975 KB

3.7 MB

3.9x

~289 bytes

100,000

11.4 MB

38.8 MB

3.4x

~288 bytes

The Bag uses about 3-5x more memory than a plain dict. The per-item overhead is consistent at around 290 bytes per node. This overhead includes:

  • The BagNode object itself

  • Internal references (parent, label, value)

  • The empty attributes dict (created on demand)

  • List slot in the parent Bag

With Attributes

10,000 items with 3 attributes each:
  Bag: 5.65 MB
  Per-item (with attrs): 592 bytes/item

Adding attributes approximately doubles the per-item memory usage (~590 bytes vs ~290 bytes). Each attribute adds to the node’s attribute dictionary.

Nested Structures

1,000 items nested 3 levels deep:
  Bag: 382 KB
  Per-item: 392 bytes/item

Nested paths create intermediate Bag nodes, adding ~100 bytes overhead per nesting level.

Memory Optimization Tips

  1. Use attributes sparingly: Each attribute adds memory overhead. Store frequently-accessed metadata in attributes, but consider keeping bulk data as values.

  2. Flatten when possible: If you don’t need the hierarchy, a flat Bag uses less memory than deeply nested paths.

  3. Consider lazy loading: Use resolvers to load large subtrees only when accessed.

  4. Clean up: Deleting nodes or calling clear() releases memory immediately.

Detailed Results

Creation

Empty Bag creation (10k): 2.29ms (0.23µs/op)
Bag from 100-key dict (1k): 150.29ms (0.15ms/op)
Bag from nested dict (1k): 10.32ms (0.01ms/op)

Creating an empty Bag is about 10x slower than creating an empty dict, but still very fast at ~0.23 microseconds. This overhead comes from initializing the internal node list.

Creating a Bag from a large dict takes longer because each key-value pair becomes a BagNode with its own metadata. For deeply nested dicts, the overhead is relatively lower.

Access Patterns

Direct access (1k keys): 0.60ms (0.60µs/op)
Nested access 5 levels (10k): 11.59ms (1.16µs/op)
Index access #0 (10k): 5.91ms (0.59µs/op)
Attribute access (10k): 6.16ms (0.62µs/op)

Direct access (bag['key']) performs well at ~0.6 µs per operation.

Nested path access (bag['a.b.c.d.e']) takes about twice as long because the path must be parsed and traversed. For performance-critical code accessing the same deep path repeatedly, consider caching a reference to the nested Bag.

Index access (bag['#0']) and attribute access (bag['key?attr']) have similar performance to direct access.

Modification

Simple assignment (10k): 19.86ms (1.99µs/op)
Nested assignment (1k): 1.98ms (1.98µs/op)
set_item with attrs (10k): 21.40ms (2.14µs/op)
Update existing key (10k): 9.17ms (0.92µs/op)

New key assignment takes ~2 µs because it creates a new BagNode.

Updating an existing key is faster (~0.9 µs) because the node already exists.

set_item with attributes has minimal overhead compared to simple assignment.

Iteration

Node iteration (100x1k): 3.16ms (0.03ms/iter)
keys() (100x1k): 0.91ms (0.01ms/iter)
values() (100x1k): 2.18ms (0.02ms/iter)
items() (100x1k): 3.48ms (0.03ms/iter)

Iteration is very efficient. Iterating over 1000 nodes takes only ~30 microseconds for keys() and ~35 microseconds for items().

Serialization

to_xml (100x100 nodes): 17.59ms (0.18ms/op)
  XML size: 3875 bytes
from_xml (100x): 60.42ms (0.60ms/op)

to_tytx JSON (100x100 nodes): 9.35ms (0.09ms/op)
  TYTX JSON size: 5757 bytes
from_tytx JSON (100x): 28.98ms (0.29ms/op)

to_tytx MsgPack (100x100 nodes): 5.83ms (0.06ms/op)
  TYTX MsgPack size: 3503 bytes
from_tytx MsgPack (100x): 30.21ms (0.30ms/op)

XML is the most verbose format but has good compatibility.

TYTX JSON is about 2x faster for serialization than XML. The output is larger because it includes type information for perfect round-trip fidelity.

TYTX MessagePack is the fastest and most compact option:

  • 3x faster serialization than XML

  • ~40% smaller than JSON

  • Requires the msgpack package

Recommendation: Use TYTX MessagePack for internal storage and network transfer. Use XML when interoperability with external systems is needed.

Resolvers

BagCbResolver creation (10k): 31.03ms (3.10µs/op)
Resolver access no cache (1k): 117.89ms (117.89µs/op)
  Callback invocations: 1000
Resolver access cached (10k): 1157.79ms (115.78µs/op)
  Callback invocations: 10000

Resolver overhead is dominated by the actual callback execution time. The resolver machinery itself adds minimal overhead.

Note: The cached resolver benchmark shows 10k callback invocations because the cache was not being hit in this simple test (cache_time was set but the benchmark runs faster than any reasonable cache duration would matter).

Subscriptions

Subscribe: 13.08µs
1k inserts with subscription: 2.14ms (2.14µs/op)
  Events fired: 1000
1k updates with subscription: 1.28ms (1.28µs/op)
  Events fired: 1000
1k inserts without subscription: 1.44ms (1.44µs/op)

Subscriptions add minimal overhead (~0.7 µs per operation). The event dispatch mechanism is efficient.

Large Bag Performance

100,000 Nodes

Create 100k nodes: 207.97ms (2.08µs/op)
Random access (1k on 100k): 0.81ms (0.81µs/op)
Full iteration (100k nodes): 1.37ms
len() on 100k bag (1k): 0.10ms (0.10µs/op)

Performance remains excellent with 100k nodes. Random access stays under 1 µs, and full iteration completes in 1.4ms.

1,000,000 Nodes

Create 1M nodes: 2.20s (2.20µs/op)
Sequential access (10k on 1M): 6.48ms (0.65µs/op)
Random access (10k on 1M): 10.29ms (1.03µs/op)
Update existing (10k on 1M): 10.19ms (1.02µs/op)
len() on 1M bag (100x): 0.01ms (0.00ms/op)
Partial iteration (100k of 1M): 1.58ms
Full iteration (1M nodes): 0.01s

Even with 1 million nodes:

  • Access time stays around 1 µs per operation

  • Full iteration completes in 10ms

  • len() is essentially instant (O(1))

The Bag scales linearly with size, maintaining consistent per-operation performance.

Comparison with Flat dict

Empty dict creation (10k): 0.16ms (0.02µs/op)
dict() from 100-key dict (1k): 0.21ms (0.21µs/op)
Dict assignment (10k): 0.84ms (0.08µs/op)
Dict access (10k): 0.80ms (0.08µs/op)

Operation

dict

Bag

Ratio

Empty creation

0.02 µs

0.23 µs

11x

Assignment

0.08 µs

2.0 µs

25x

Access

0.08 µs

0.6 µs

7x

A Bag is naturally slower than a plain dict because it provides much more functionality: hierarchical paths, attributes, subscriptions, resolvers, and serialization. The overhead is acceptable for most applications.

Comparison with Hierarchical Structures

A more meaningful comparison is with other hierarchical data structures: nested dicts and xml.etree.ElementTree from the standard library.

Nested dict

Nested dict creation (100x1k items, 3 levels): 12ms (0.12ms/op)
Nested dict access (10k): 0.29ms (0.03µs/op)
Bag creation (100x1k items, 3 levels): 216ms (2.16ms/op)
Bag access (10k): 8.80ms (0.88µs/op)

Operation

nested dict

Bag

Ratio

Creation (1k items)

0.12 ms

2.16 ms

18x

Access

0.03 µs

0.88 µs

29x

Nested dicts are faster because they’re the native Python structure. However:

  • Nested dict requires manual key checking: if 'level1' not in d: d['level1'] = {}

  • Bag creates intermediate nodes automatically: bag['a.b.c'] = value

  • Bag provides iteration, attributes, subscriptions, and serialization

xml.etree.ElementTree

ElementTree creation (100x1k items): 17ms (0.17ms/op)
ElementTree find (10k): 49ms (4.89µs/op)
ElementTree index access (10k): 0.40ms (0.04µs/op)
ElementTree attrib access (10k): 0.39ms (0.04µs/op)

Bag path access (10k): 9.36ms (0.94µs/op)
Bag attribute access (10k): 9.89ms (0.99µs/op)

Operation

ElementTree

Bag

Winner

Creation

0.17 ms

2.16 ms

ET (13x faster)

XPath-like find

4.89 µs

0.94 µs

Bag (5x faster)

Index access

0.04 µs

0.94 µs

ET (24x faster)

Attribute access

0.04 µs

0.99 µs

ET (25x faster)

Key insights:

  • ElementTree index access (root[0][0][500]) is very fast but requires knowing the structure

  • ElementTree find (XPath search) is slower than Bag’s path syntax

  • Bag path access (bag['level1.level2.item500']) is predictable and readable

  • ElementTree requires string conversion for all values; Bag preserves Python types

Serialization

ElementTree tostring (100x100 items): 12ms (0.12ms/op)
  Size: 4525 bytes
ElementTree fromstring (100x): 5ms (0.05ms/op)

Bag to_xml (100x100 items): 17ms (0.17ms/op)
  Size: 3875 bytes
Bag from_xml (100x): 56ms (0.56ms/op)

Operation

ElementTree

Bag

Ratio

Serialize

0.12 ms

0.17 ms

1.4x

Deserialize

0.05 ms

0.56 ms

11x

Output size

4525 bytes

3875 bytes

Bag 14% smaller

ElementTree serialization is faster, but Bag’s XML is more compact because it encodes values as attributes rather than text nodes. For faster serialization, use TYTX MessagePack (0.06 ms).

Memory

1000 items nested 3 levels deep:
  Nested dict: 95 KB
  ElementTree: 169 KB
  Bag: 361 KB

Ratios (vs nested dict):
  ElementTree: 1.8x
  Bag: 3.8x

Bag uses more memory because each node carries:

  • Value (any Python type)

  • Attributes dict

  • Parent reference

  • Label

  • Optional resolver

This is the cost of rich functionality.

When to Use Each

Use Case

Recommended

Simple nested config

nested dict

XML parsing/generation

ElementTree

Hierarchical data with attributes

Bag

Need subscriptions/reactivity

Bag

Need lazy loading (resolvers)

Bag

Multiple serialization formats

Bag

Type-preserving round-trips

Bag

When to use dict instead: If you need a simple key-value store with millions of operations per second and none of the Bag features, use a plain dict.

When to use Bag: For hierarchical data with metadata, lazy loading, change tracking, or serialization needs, the Bag’s features justify its overhead.

Performance Tips

  1. Cache deep paths: If you access bag['a.b.c.d.e'] repeatedly, store the result in a variable.

  2. Use TYTX MessagePack: For serialization, it’s faster and more compact than XML or JSON.

  3. Batch operations: When making many changes, consider building a dict first and creating the Bag once.

  4. Subscriptions are cheap: Don’t hesitate to use them; the overhead is minimal.

  5. Resolvers cache by default: Take advantage of cache_time to avoid repeated expensive operations.