Scalability Analysis of personal_world.SetChunkVerifier on MAP State

TL;DR

• SetChunkVerifier is highly sensitive to the size of the map stored within a world.
• It works at N=10000, but at N=100000 the very first call fails with an out-of-gas error on CPUCycles.
• CPU cost grows much faster than storage usage.
• Even in the precondition batches, gas_per_item increases by about 10x, which indicates a nonlinear cost pattern.

Conclusion: In the current structure, large-scale maps are not practical. CPU gas becomes the first bottleneck well before storage does.

The full dataset is available here:

• Raw Milestone Data
• Raw Failure Data

Structure Summary

This experiment does not use a simple flat map. Instead, it uses the following layered structure:

• Outer layer: AVL(worldID -> value)
• Inner value: map[string]string

SetChunkVerifier works as follows:

1. Look up the verifier map from the outer AVL using worldID.
2. Add or overwrite a value in map[coordKey]verifier for that world.

Experiment setup:

• A single verifier world is fixed throughout the test.
• The number of entries in the map for that same world is continuously increased.

In other words, N in this experiment refers to the size of a single map, not the number of worlds.

Experiment Environment

• Runtime: custom gnodev build
• Branch: dev/jae/gas-model-improvements
• Commit: 2d507b86935576c6eb5b06cae3b14861f0090c87

Result Summary (Milestones)

At 100k, even a single call fails immediately, which confirms that CPU cost is the limiting factor.

Key Observations

1. CPU Becomes the Bottleneck Before Storage

• storage_per_item remains nearly constant.
• gas_per_item keeps increasing.
• The out-of-gas error is caused by CPUCycles.

This indicates that the system reaches its limit because of CPU cost, not because of storage writes.

2. Cost Growth Is Nonlinear

Comparison across precondition batches:

• gas_per_item increases by about 10.7x
• elapsed_sec increases by about 6x
• storage_per_item shows almost no change

As the map grows, the processing cost does not increase linearly.

3. Performance Degrades During the Precondition Phase

• Batch cost keeps increasing even before the single-call measurement begins.
• This suggests the issue starts with growth in the map itself, not only at the moment of read or write.

Precondition Generation Method

• Batch size: 200
• Final batch size: 190
• Call method: SetChunkVerifiers (batch)

Precondition Batch Trend

Even with the same batch size, the cost continues to rise over time.

The full batch history is available here:

• Raw Milestone Data

Estimated Store Calls Per Single Invocation

Based on static analysis, not VM instrumentation

• StoreGetObject: [hypothesis] 4..7
• StoreSetObject: [hypothesis] 4..7

Reasoning:

• worldID lookup in the AVL
• loading and modifying the map object
• storing dirty objects and the owner chain

The dominant factor appears to be the growing internal map operations and serialization cost rather than the raw number of objects.

Parameter Sizes

Approximate payload estimates based on the function contract and key normalization logic

SetChunkVerifier(worldID, chunkKey, verifier) accepts three logical inputs, but the stored payload is slightly smaller than the request payload because chunkKey is normalized before insertion.

• worldID

  • Logical type: uint32
  • About 4 bytes as a numeric value
  • Used to look up the outer AVL entry for the world
  • Not duplicated in every inner map entry

• chunkKey

  • Format: "<worldID>:0_<index>"
  • Includes the world prefix in the request
  • Normalized before storage, so only the coordinate portion ("0_<index>") is kept in the inner map
  • Rough request size: about 5~10 bytes
  • Rough stored key size after normalization: about 3~8 bytes

• verifier

  • Measurement value: "measure_v" (9 bytes)
  • Precondition value: "v_<i>" (3~8 bytes)

Approximate Size Per Set

• Request payload per SetChunkVerifier call:

  • approximately 12~23 bytes of raw logical content
  • calculated as worldID (~4 bytes) + chunkKey (510 bytes) + verifier (39 bytes)

• Stored payload per inner map entry:

  • approximately 6~17 bytes of raw string content
  • calculated as normalized coordKey (38 bytes) + verifier (39 bytes)

• Per-world shared key:

  • the outer AVL stores worldID once per world as a separate lookup key

This means the per-entry stored payload is relatively small. The main issue is unlikely to be the raw byte size of each value itself, and is more likely tied to map growth, object serialization, and repeated internal processing as the map becomes larger.

Conclusion

personal_world.SetChunkVerifier shows the following characteristics:

• It is highly sensitive to map size growth.
• CPU cost grows much faster than storage cost.
• A practical limit is already reached around 100k.
• Cost accumulation begins during the precondition phase.

Overall, as map size increases, CPU cost rises quickly enough to make large-scale map usage impractical in the current structure. CPU gas becomes the bottleneck before storage does.