Horizon 1.0.0

Before you upgrade

• If you were using the new ingestion in one of the previous versions of Horizon, you must first remove ENABLE_EXPERIMENTAL_INGESTION feature flag and restart all Horizon instances before deploying a new version.
• The init stage (state ingestion) for the public Stellar network requires around 1.5GB of RAM. This memory is released after the state ingestion. State ingestion is performed only once. Restarting the server will not trigger it unless Horizon has been upgraded to a newer version (with an updated ingestion pipeline). It's worth noting that the required memory will become smaller and smaller as more of the buckets in the history archive become CAP-20 compatible. Some endpoints are not available during state ingestion.
• The CPU footprint of the new ingestion is modest. We were able to successfully run ingestion on an AWS c5.xlarge instance. The init stage takes a few minutes on c5.xlarge. c5.xlarge is the equivalent of 4 vCPUs and 8GB of RAM. The definition of vCPU for the c5 large family in AWS is the following:

The 2nd generation Intel Xeon Scalable Processors (Cascade Lake) or 1st generation Intel Xeon Platinum 8000 series (Skylake-SP) processor with a sustained all core Turbo frequency of up to 3.4GHz, and single core turbo frequency of up to 3.5 GHz.

• The state data requires an additional 6GB DB disk space for the public Stellar network (as of February 2020). The disk usage will increase when the number of Stellar ledger entries increases.
  • accounts_signers table: 2340 MB
  • trust_lines table: 2052 MB
  • accounts table: 1545 MB
  • offers table: 61 MB
  • accounts_data table: 15 MB
  • exp_asset_stats table: less than 1 MB
• A new environment variable (or command line flag) needs to be set so that Horizon can ingest state from the history archives:
  • HISTORY_ARCHIVE_URLS="archive1,archive2,archive3" (if you don't have your own pubnet history archive, you can use one of SDF's archives, for example https://history.stellar.org/prd/core-live/core_live_001)
• Horizon serves the endpoints /paths and /order_book from an in-memory graph, which is only available on ingesting instances. If some of the instances in your cluster are not configured to ingest, you can configure your proxy server to route those endpoints to the ingesting instances. This is beyond the scope of this document - consult the relevant documentation for your proxy server. A better solution for this will be released in the next Horizon version.

New Ingestion System

The most substantial element of this release is a full rewrite of Horizon's ledger ingestion engine, which enables some key features:

• A set of important new endpoints (see below). Some of these were impossible under the previous ingestion architecture.
• An in-memory order book graph for rapid querying.
• The ability to run parallel ingestion over multiple Horizon hosts, improving service availability for production deployments.

The new engine resolves multiple issues that were present in the old system. For example:

• Horizon's coupling to Stellar-Core's database is dramatically reduced.
• Data inconsistency due to lag between endpoints is eliminated.
• Slow endpoints (path-finding for example) are now speedy.

Finally, the rearchitecting makes new reliability features possible. An example is the new internal state verifier, which guarantees consistency between the local Horizon state and the public history archives.

The admin guide contains all the information needed to operate the new ingestion system.

Added

• Add /accounts endpoint, which allows filtering accounts that have a given signer or a trustline to an asset.
• Add /offers endpoint, which lists all offers on the network and allows filtering by seller account or by selling or buying asset.
• Add /paths/strict-send endpoint, which enables discovery of optimal "strict send" paths between assets.
• Add /paths/strict-receive endpoint, which enables discovery of optimal "strict receive" paths between assets.
• Add the fields max_fee and fee_charged to /fee_stats.

Breaking changes

Changed

• Change multiple operation types to their canonical names for operation resources (#2134).

• Change the type of the following fields from number to string:

  • Attribute offer_id in manage buy offer and manage sell offer operations.
  • Attribute offer_id in Trade effect.
  • Attribute id in Offer resource.
  • Attribute timestamp and trade_count in Trade Aggregation resource.

  See #1609, #1909 and #1912 for more details.

Removed

• /metrics endpoint is no longer part of the public API. It is now served on ADMIN_PORT/metrics. ADMIN_PORT can be set using env variable or --admin-port CLI param.

• Remove the following fields from /fee_stats:

  • min_accepted_fee
  • mode_accepted_fee
  • p10_accepted_fee
  • p20_accepted_fee
  • p30_accepted_fee
  • p40_accepted_fee
  • p50_accepted_fee
  • p60_accepted_fee
  • p70_accepted_fee
  • p80_accepted_fee
  • p90_accepted_fee
  • p95_accepted_fee
  • p99_accepted_fee

• Remove fee_paid field from Transaction resource (Use fee_charged and max_fee fields instead - see #1372).

Horizon 1.0.0 Beta

SDF unstable repository package name: stellar-horizon=1.0.0-beta~unstable-674

New Ingestion System

This beta release previews the first major release of Horizon. This release is NOT recommended for production deployments. It is suitable for testing setups, staging servers, and developer use. This release allows Horizon operators to become familiar with Horizon's significant new features, and try them out in their own setups, in advance of the stable release.

The most substantial element of this release is a full rewrite of Horizon's ledger ingestion engine, which enables some key features:

• A set of important new endpoints (see below). Some of these were impossible under the previous ingestion architecture.
• An in-memory order book graph for rapid querying.
• The ability to run parallel ingestion over multiple Horizon hosts, improving service availability for production deployments.

The new engine resolves multiple issues that were present in the old system. For example:

• Horizon's coupling to Stellar-Core's database is dramatically reduced.
• Data inconsistency due to lag between endpoints is eliminated.
• Slow endpoints (path-finding for example) are now speedy.

Finally, the rearchitecting makes new reliability features possible. An example is the new internal state verifier, which guarantees consistency between the local Horizon state and the public history archives.

The testing guide contains all the information needed to start testing the new ingestion system.

Added

• Add /accounts endpoint, which allows filtering accounts that have a given signer or a trustline to an asset.
• Add /offers endpoint, which lists all offers on the network and allows filtering by seller account or by selling or buying asset.
• Add /paths/strict-send endpoint, which enables discovery of optimal "strict send" paths between assets.
• Add /paths/strict-receive endpoint, which enables discovery of optimal "strict receive" paths between assets.
• Add the fields max_fee and fee_charged to /fee_stats.

Breaking changes

Changed

• Change multiple operation types to their canonical names for operation resources (#2134).

• Change the type of the following fields from number to string:

  • Attribute offer_id in manage buy offer and manage sell offer operations.
  • Attribute offer_id in Trade effect.
  • Attribute id in Offer resource.
  • Attribute timestamp and trade_count in Trade Aggregation resource.

  See #1609, #1909 and #1912 for more details.

Removed

• Remove the following fields from /fee_stats:

  • min_accepted_fee
  • mode_accepted_fee
  • p10_accepted_fee
  • p20_accepted_fee
  • p30_accepted_fee
  • p40_accepted_fee
  • p50_accepted_fee
  • p60_accepted_fee
  • p70_accepted_fee
  • p80_accepted_fee
  • p90_accepted_fee
  • p95_accepted_fee
  • p99_accepted_fee

• Remove fee_paid field from Transaction resource (Use fee_charged and max_fee fields instead - see #1372).

Horizon 1.0.0 Alpha

New Ingestion System

This alpha release previews the first major release of Horizon. This release is NOT recommended for production deployments. It is suitable for testing setups, staging servers, and developer use. This release allows Horizon operators to become familiar with Horizon's significant new features, and try them out in their own setups, in advance of the stable release.

The most substantial element of this release is a full rewrite of Horizon's ledger ingestion engine, which enables some key features:

• A set of important new endpoints (see below). Some of these were impossible under the previous ingestion architecture.
• An in-memory order book graph for rapid querying.
• The ability to run parallel ingestion over multiple Horizon hosts, improving service availability for production deployments.

The new engine resolves multiple issues that were present in the old system. For example:

• Horizon's coupling to Stellar-Core's database is dramatically reduced.
• Data inconsistency due to lag between endpoints is eliminated.
• Slow endpoints (path-finding for example) are now speedy.

Finally, the rearchitecting makes new reliability features possible. An example is the new internal state verifier, which guarantees consistency between the local Horizon state and the public history archives.

The testing guide contains all the information needed to start testing the new ingestion system.

Added

• Add /accounts endpoint, which allows filtering accounts that have a given signer or a trustline to an asset.
• Add /offers endpoint, which lists all offers on the network and allows filtering by seller account or by selling or buying asset.
• Add /paths/strict-send endpoint, which enables discovery of optimal "strict send" paths between assets.
• Add /paths/strict-receive endpoint, which enables discovery of optimal "strict receive" paths between assets.
• Add the fields max_fee and fee_charged to /fee_stats.

Breaking changes

Changed

• Change multiple operation types to their canonical names for operation resources (#2134).

• Change the type of the following fields from number to string:

  • Attribute offer_id in manage buy offer and manage sell offer operations.
  • Attribute offer_id in Trade effect.
  • Attribute id in Offer resource.
  • Attribute timestamp and trade_count in Trade Aggregation resource.

  See #1609, #1909 and #1912 for more details.

Removed

• Remove the following fields from /fee_stats:

  • min_accepted_fee
  • mode_accepted_fee
  • p10_accepted_fee
  • p20_accepted_fee
  • p30_accepted_fee
  • p40_accepted_fee
  • p50_accepted_fee
  • p60_accepted_fee
  • p70_accepted_fee
  • p80_accepted_fee
  • p90_accepted_fee
  • p95_accepted_fee
  • p99_accepted_fee

• Remove fee_paid field from Transaction resource (Use fee_charged and max_fee fields instead - see #1372).

horizonclient v2.0.0 & txnbuild v2.0.0

• Add custom UnmarshalJSON() implementations to Horizon protocol structs so int64 fields can be parsed as JSON numbers or JSON strings
• Remove deprecated fee_paid field from Transaction response
• Dropped support for Go 1.10, 1.11.

Horizon v0.24.1

Add

• Add cache to improve performance of experimental ingestion system (#2004).

Fix

• Fix experimental ingestion bug where ledger changes were not applied in the correct order (#2050).
• Fix experimental ingestion bug where unique constraint errors are incurred when the ingestion system has to reingest state from history archive checkpoints (#2055).
• Fix experimental ingestion bug where a race condition during shutdown leads to a crash (#2058).

Horizon v0.24.0

Add

• Add fee_charged and max_fee objects to /fee_stats endpoint (#1964).
• Experimental ledger header ingestion processor (#1949).
• Improved performance of asset stats processor (#1987).
• Provide mechanism for retrying XDR stream errors (#1899).
• Emit error level log after 3 failed attempts to validate state (#1918).
• Fixed out of bounds error in ledger backend reader (#1914).
• Fixed out of bounds error in URL params handler (#1973).
• Rename OperationFeeStats to FeeStats (#1952).
• All DB queries are now cancelled when request is cancelled/timeout. (#1950).
• Fixed multiple issues connected to graceful shutdown of Horizon.

Scheduled Breaking Changes

• All *_accepted_fee fields in /fee_stats endpoint are deprecated. These fields will be removed in Horizon 0.25.0.

• The following operation type names have been deprecated:

  • path_payment
  • manage_offer
  • create_passive_offer.

  The names will be changed in 0.25.0 to:

  • path_payment_strict_receive
  • manage_sell_offer
  • create_passive_sell_offer

• fee_paid field on Transaction resource has been deprecated and will be removed in 0.25.0. Please use new fields added in 0.18.0: max_fee that defines the maximum fee the source account is willing to pay and fee_charged that defines the fee that was actually paid for a transaction. See CAP-0005 for more information.

• The type for the following attributes will be changed from int64 to string in 0.25.0:

  • Attribute offer_id in manage buy offer and manage sell offer operations.
  • Attribute offer_id in Trade effect.
  • Attribute id in Offer resource.
  • Attribute timestamp and trade_count in Trade Aggregation resource.

Check Beta Testing New Ingestion System if you want to test the new ingestion system.

Compliance v0.0.33

• Add ReadTimeout to HTTP server configuration to fix potential DoS vector.
• Dropped support for Go 1.10, 1.11.

Bridge v0.0.33

• Add ReadTimeout to HTTP server configuration to fix potential DoS vector.
• Fixed path-payment operation in /payment
• Dropped support for Go 1.10, 1.11.

Ticker v1.2.0

• Add ReadTimeout to Ticker HTTP server configuration to fix potential DoS vector.
• Added nested "issuer_detail" field to /assets.json.
• Dropped support for Go 1.10, 1.11.

Horizon v0.23.1

• Add ReadTimeout to Horizon HTTP server configuration to fix potential DoS vector.