When considering blockchain technology for data integrity—particularly in the realm of log file immutability—there are several critical barriers. Without addressing each of these potential barriers the solution will ultimately fail, either outright or by lack of adoption, because each one can render the solution unusable, unscalable, or both.
Performance – Application log files will often produce millions of line items per hour and organizations are unwilling to slow performance even for the benefit of security. As such, any solution that interacts or utilizes the log data needs to be built for high throughput environments capable of tens to hundreds of thousands of transactions per second. This type of performance will allow for synchronous or near-synchronous digital replication of the log files into an immutable structure.
Storage – Standard blockchain architecture makes immutability synonymous with perpetual storage, meaning data committed to the chain will be there forever. This is problematic when dealing with application log files that produce millions of unique data points per hour. One could see how the length of the chain could quickly get out of hand, bringing about additional consequences such as degraded performance, increased resource demands, and exorbitant costs.
Cost – Many blockchains have high, at least in relative terms, per transaction costs. For example, given the transaction costs of Ethereum and Bitcoin as of the writing of this post (late October 2021), and assuming a log validation occurs every fifteen minutes, or ninety-six times per day, and assuming the each is committed to both the Ethereum chain and the Bitcoin chain, the cost for the Ethereum solution would be $16,162, and the Bitcoin solution would be $752,860 per month, just for the blockchain storage for a single blockchain. This is fractional when compared to high value transactions, but in the case of log files being committed to the chain it quickly becomes prohibitive.
Resources – One method often perceived as an alternative to traditional blockchains is hosting a private blockchain. However, the compute and resource requirements for private chains built in traditional ways remain prohibitively high, negating the viability of such a plan as a cost-saving measure. Additionally, self-hosting a blockchain solution counteracts some of the benefits and key features of blockchain.
Introducing Polymer Proof
Polymer Proof is a multi-use data integrity solution built on our proprietary form of blockchain called Polymer Chain. Polymer Chain is a lightweight, highly-performant, energy efficient form of blockchain technology designed for high throughput environments. The solution is further differentiated from most other blockchain solutions with its inclusion of an authentication layer, as well as an intelligent immutability, which distinguishes immutability from “forever storage.” By including intelligently immutable architecture, data can be securely removed or archived when it is no longer needed, such as the end of the operational lifetime of the data.
Polymer Proof is designed and has been tested to process up to 1.2M transactions per second (TPS), and with additional resiliency layers was tested up to 150,000 TPS. The solution is horizontally scalable to allow for “infinite” scalability regarding disparate solutions or chains. The data is striped across multiple database structures on the hosting service creating a decentralized data store.
Covax Data’s approach is to have key data points replicated to a high-performance database(s) co-resident with Polymer Chain, not the client application. This database would be populated for a fixed period, at which time a hash would be taken of the database representing the immutable digital twin of the log, a hash would be taken of the local log files for the application being secured, with both hashes being compared and committed to the blockchain. If the hashes match—indicating that they are absent of any tampering—the replica could be wiped and reset, keeping storage costs minimal. If the hashes do not match an alert would be raised and the replica database(s) preserved for investigation.
Polymer Proof and Polymer Chain are cloud deployed software-based solutions. Leveraging cloud deployments brings increased scale and security to the platform, and as the diversification is broadened across multiple clouds, the scalability, security, and resiliency of the platform will be significantly increased. This deployment model also allows for platform resources to easily be deployed as microservices, creating an environment that benefits from high resource availability without the cost of those services always being plugged in, therefore dramatically reducing both the financial and resource cost of the implementation.
While technically the hashes that are being committed to the blockchain could be committed to any blockchain, thus being blockchain agnostic, it fails to be a viable option at the current cost levels of most blockchains. The Polymer Proof solution starts as low as $250 per month.
Beginning next week, we will take a closer look at individual approaches to using Polymer Proof to solve various data integrity challenges.