[Proposal] - Improving Protocol Security: on-chain realtime hack prevention

In the following proposal to the Curve community, we want to elaborate on what has happened in the most recent hack related to the Vyper compiler and how future exploits based on external attack vectors can be prevented. Furthermore, we are going to look into the underlying, fundamental characteristics of hacks, as well as looking into designing a safer security system based on identifying those characteristics on-chain, thereby preventing exploits in realtime.

Our team has been intensively working for the last 6 months on tangible solutions to mitigate the most profound attack vectors of DeFi protocols. We believe to have found a realistic and suitable solution with our most recent development: On-Chain Firewalls.

We are looking forward to opening up a discussion on the governance forum around our proposed security improvements to Curve. We hope to gain feedback, suggestions and potential support that could lead to a grant for further developing this novel concept.


The “unknown unknowns” of mitigating protocol risk

The most recent exploit on Curve is a perfect example of how dependencies, outside the protocols actionable control, can become attack vectors, thereby compromising the overall protocols security.

Audits would not be even able to detect that vulnerability. Problem was lying in a deeper layer of the stack.

However most auditors assume exactly that: the code that the compiler produces is exactly the code that the project intends to use, for the purposes of auditing.

Basically, assume the compiler is "correct", and it's the user using the language incorrectly if there's a problem

— señor doggo 🏴🏴‍☠️ in his wartime ceo era (@fubuloubu) July 31, 2023

https://twitter.com/fubuloubu/status/1685856295033659392?ref_src=twsrc^tfw

Most established exploit prevention procedures like performing security audits are important, yet, we must understand that these have limitations and cannot asses risks beyond code produced by the engineering team. External dependencies as potential attack vectors are frequently being disregarded or assumed to be safe, as they lies outside a single protocol’s spectrum of actionable security measures. Furthermore, given the example of the most recent curve exploit: Auditing legacy versions of the Vyper compiler, was undoubtably out of the protocol’s security scope.

no_firewall

Even when exploring hypothetical scenarios, e.g.: All protocols using Vyper coming together beforehand and collectively sourcing audits for Vyper versions 0.2.15, 0.2.16, and 0.3.0, it still would not have solved the underlying problem: The unknown (un)knowns of vulnerable external dependencies. Given the infinite amount of unknowns, it is merely impossible to effectively draw the line between where to start and stop security audits.

no_firewall_cycle

This is why protocols have to hope for the best while keeping their fingers crossed🤞once deploying code to production. In essence: Hoping they won’t have to face any unknown (un)knowns in production, but if so, hopefully only through bug bounties programs and whitehats. Usually, the financial incentives for blackhat hackers largely outweigh bug bounty initiatives, leading to exploits of undiscovered attack vectors. Furthermore, in accordance to Murphy’s Law, the above strategy would lead to a hack in any case, sooner or later.


Actionable opportunities to protect the protocol

Adding Firewalls to the protocols production deployments

As an additional layer of protection to the protocol Firewalls can be added to mitigate in-production risks.

firewall_cycle

Even tough, the way the protocol gets exploited relies on an unknown (un)known, the target of the exploit always is a KNOWN KNOWN, being liquidity.

unknown

→ Therefore, we can work with the known characteristics of every exploit happening, instead of trying to solve what we cannot know.

known

We are aware of the attackers target: extracting Liquidity.

We are aware of external dependencies, will most likely lead the attacker to his target.

→ knowing and understanding these two crucial pieces of information, we can design a security system around these two vectors.

firewall

As a wrapped layer of security around the protocol, the attacker will not be able to reach his intended target: extracting liquidity, with external dependencies as his attack vector. Even when vulnerabilities occur (e.g. faulty compilers), an attacker will not reach his target, as the Firewall system operates as an independent layer of security for the protocol.

Whenever abnormal activity occurs, on-chain Firewall’s offer a range of protective measures, including blocking of transactions, pausing the protocol, freezing of funds, or controlled fund withdrawal, thereby providing a robust on-chain defence against potential hacks.

Protective Measures

Once the Firewall has been triggered, protocols can use it in a variety of ways to protect their assets. Here are some of the protective measures that can be taken:

firewall_triggered

  1. Block the exploit transaction: The protocol can choose to block the transaction that caused the liquidity drain. This is particularly useful in preventing unauthorized transactions that could potentially lead to theft of funds.
  2. Pause the protocol: In some cases, the protocol might decide to pause all operations until the issue that caused the Firewall to be triggered is resolved. This is a more drastic measure that can be used when the integrity of the entire protocol is at risk.
  3. Freeze affected funds: If the transaction that triggered the Firewall involves a withdrawal from the DeFi protocol, the protocol can freeze the funds affected by the transaction. This prevents the funds from being withdrawn until the issue is resolved. The funds can then be released upon the decision from a trusted third party for settling the dispute.
  4. Throttle Fund Withdrawal: In some cases, the protocol might choose to throttle the withdrawal of funds to a smaller amount over time. This can be particularly useful in preventing large, suspicious withdrawals that could be indicative of a hack.

How it works in detail

In the following we describe our approach to creating an on-chain security layer for tracking and identifying malicious activity in realtime, on a blockchain level. This would allow for the creation of on-chain Firewalls, thereby allowing exploits to be prevented in realtime.

:bulb: TL;DR
The on-chain Firewall works by allowing protocols to specify a security parameter (number) that may not change by more than a given amount, in a given time window. Using the TVL as the security parameter would effectively enable protocols to track huge and sudden TVL fluctuations, which are highly indicative of malicious activity, directly on-chain.

Parameter-Based System

The parameter-based design of the system is the cornerstone of the Firewall’s effective operation. This system allows any protocol to define a configuration for their Firewall.

The protocol can configure the Firewall with a threshold percentage (for the parameter), a block interval (time period), and a starting parameter. The parameter, can then be updated at any time. Whenever the current parameter changes by more than the set threshold, the Firewall gets triggered. Protocol’s may individualistically decide upon fitting protective measures that are initiated upon a Firewall trigger.

Parameter as a Tracking Tool

The security parameter (a number) serves as a tracking tool. In a simple implementation, this parameter could be the Total Value Locked (TVL) of a protocol. By using the TVL as the parameter, the system can monitor the volume of withdrawals from the protocol. This is particularly useful in preventing large, sudden withdrawals within a short time frame, which are often indicative of malicious activity or a potential hack. Most protocols experience a significant withdrawal within a single transaction when being hacked, and parameter-based Firewall systems can help prevent that from happening.

Versatility and Composability

One of the key strengths of the parameter-based system is its high versatility and composability. Protocol’s can choose what they want to track and use to identify malicious activity. While TVL is a common choice for tracking large withdrawals, protocols are not limited to this. They could choose to track other parameters that are indicative of hacks, depending on their specific needs and circumstances. This allows for highly individualistic configurations for each protocol, making it adaptable to a wide range of scenarios.In essence, the parameter-based system provides a flexible and versatile solution for protecting DeFi protocols.

Protocol’s may choose to use one or multiple security parameter(s). Each security parameter is referenced by its creator (an address) and a unique identifier. These identifiers may be a token address, a hash, or any arbitrary Solidity bytes32 value. Said approach maximizes compatibility, by practically allowing protocol’s to track metrics for specific ERC20 tokens or other unique identifiers separately.

Nevertheless, this approach should not be confused with using multiple parameters (values) to compute a single security parameter. A security parameter may also be the result of a mathematical computation combining a set of multiple values.

Configuration parameters in detail

The Firewall configuration can be set up by the protocol governance / owner, by defining a set of rules for the individual Firewall implementation. The combination of these rules allow for highly individualistic and dynamic Firewall implementations.

Defining configuration parameters

  • start parameter: Is the starting security parameter for a given protocol’s security parameter.
  • threshold percentage: Defines the maximum limit, which a parameter is allowed to change in a given time window.
  • block interval: Defines the number of blocks (time window) within the parameter should not change by more than the threshold percentage.
  • Optional: The cooldown period : Defines the period of time a protocol must wait before they can change their security parameters again after the Firewall has been activated.

NOTE: If storing multiple security parameters, configuration has to be done for each security parameter separately.


Quantitative measures

Total liquidity protected:

  • D_i = deposit at block i
  • W_i = withdrawal at block i
  • T_i = threshold at block i
  • F_i = Firewall triggered at block i (1 if the Firewall is triggered at block i, 0 if the Firewall is not triggered)
  • n = Time windows for the Firewall as a number of blocks (block interval over which the threshold is computed)
  • m = number of blocks for the cooldown period

Cooldown function at block j

This function checks whether a block j is within the cooldown period after the Firewall has been triggered. It returns 1 if the Firewall is triggered, 0 otherwise.

  • CD(j) = max_{k=1}^{j-1} (F_k * I(j - k <= m))

where I(condition) is the indicator function, returning 1 if the condition is true and 0 otherwise.

Liquidity function at block i:

If the block i is within the cooldown period (CD(i) = 1), the withdrawal term W_i * (1 - CD(i)) will be zero.

  • L_i = (D_i - W_i * (1 - CD(i))) * T_i

Total Liquidity protected by the Firewall at block i:

The total liquidity protected over the n blocks is the sum of the liquidity protected at each block:

  • TotalLiquidityProtected = sum_{i=1}^n L_i


Implementation and Integration

The parameter-based Firewall system we are aiming for, is designed to be easily integrated into existing DeFi protocols. Its flexible and adaptable nature allows it to work seamlessly with a wide range of systems. Overall, there are three general steps to integration:

  1. Define configuration parameters: The first step is to define the parameters for your Firewall. This includes the starting parameter, the time-period, and the threshold. The choice of parameters will depend on what you want to track and the specific needs of your protocol.
  2. Integrate: Once the configuration values have been set, protocol’s can implement the on-chain Firewall. This includes minor code manipulation within the protocol’s codebase.
  3. Monitor and Adjust: After integration, protocol’s can monitor the operation of the Firewall and adjust the configuration values as needed. This allows for fine-tuning the system to better protect the protocol.

The key to effective use of the Firewall’s is leveraging specific needs of the protocol and adjusting the configuration accordingly. With the right configuration, the parameter-based, on-chain Firewall can provide a robust line of defence against potential hacks and unauthorized fund withdrawals.

Configuration

To configure the Firewall, protocol’s have to set the configuration parameters. These configuration values can be updated at any time.

Considerations

To ensure accurate operation of the Firewall, protocol’s have to include updating the security parameter frequently (on-chain), preferably whenever a dependency of it changes within its own state in storage.

To clarify, “dependency” means a dependency sub-parameter used to compute the security parameter (if the security is the result of multiple parameters using a formula, e.g. price rates, TVL), or the sole parameter (e.g. TVL) being used for the security parameter. This can be done on a per-transaction basis or a once per block basis, to ensure gas savings.

Nevertheless, how frequently the security parameter(s) is/are updated is - at the end - up to the protocol’s implementation. Therefore, integration can be highly individualistic.

Despite the automatic activation of the Firewall, protocol’s retain control over the status of the Firewall. They can manually deactivate or activate their Firewall at any given moment using a setFirewallStatus Solidity function.

Code modifications of the protocol

In general, the system is designed to work by only requiring moderate code manipulation at the protocol itself. In essence, implementing the Firewall requires including one function call to update the security parameter(s) wherever applicable within the protocol, while also checking the Firewall status before initiating TVL manipulations on the protocol (user withdrawals).

Whenever the Firewall gets triggered, protocol’s also have multiple options on what protective measures to enact as specified before. These would also have to be configured and integrated, as well.

Nevertheless, the amount of code manipulation needed heavily depends on the parameter’s used within the protocol to indicate malicious activity. Therefore, implementation work requirements can vary.


Our Proposal for Curve

We are proposing to research and build tailored Firewall implementations to improve the security of the Curve protocol and prevent most known attack vectors by default. Furthermore, our proposal will enhance other verticals, such as governance participation, as we are introducing new governance mechanisms that will allow safe management of emergency situations through DAO votings.

The idea of on-chain Firewalls is not new. However, with the most recent developments and research findings, new opportunities have evolved. Especially for asset-heavy protocols such as Curve. Lightweight updates could exponentially increase the protocol’s security.

With our contributions in various security initiatives, our research work, and our demos at various hackathons, we managed to prove how exploits on contracts with in-production vulnerabilities could be prevented.

The two main ones have been our Decentralized Circuit Breaker @ ETHGlobal Paris 2023 and our Firewall Hack at SOZU HAUS Paris 2023. Another one has been an on-chain Firewall to protect Uniswap v3 from fraudulent contract interactions. A demo of this use-case received great acknowledgement at ETHPrague, see here.

After demonstrating the effectiveness of our approach through our prize-winning demos, and having raised enough confidence in Firewall implementations through our research and contributions, we are keen to further this exploration in collaboration with curve.

For this project, we propose to develop a novel on-chain Firewall system designed for optimal security, flexibility, and governance. This system will provide advanced measures to safeguard Curve against all sorts of external attack vectors, all the while, preserving the core decentralized ethos of Curve.

This proposed project will not necessitate any immediate changes or updates to the Curve protocol, until mainnet integration decisions are made.

Our Motivation**:**

DON’T LET THAT EVER HAPPEN AGAIN! … We believe the curve team is one of the most capable and respected teams in all DeFi. How Curve decides to react, given the current situation, has the power to set new standards for DeFi Security … This is a tremendous opportunity that we don’t get often. Let’s make sure we set a strong and innovative tone by establishing new security standards by opening a new chapter in DeFi Security. This hack should be remembered as a turning point for security in web3.

If we succeed we will have added another layer of security to Curve, protecting its assets.

Goals

  1. Develop innovative methods for proactive, on-chain protocol security within the Curve ecosystem.
  2. Enhance and secure the upcoming modifications to the Curve Protocol by integrating tailor-made Firewalls.
  3. Quantify and minimize protocol security risks through the use of Firewalls.
  4. Improve user experience by offering opt-in security features.
  5. Enhancing Governance Participation by adding new functionality that naturally:
    1. Motivates participation from individual voters
    2. Increases the number of high-quality and engaged delegate platforms
    3. Improves the visibility and understanding of governance processes and proposals
  6. Identify and overcome potential implementation challenges before the integration of Firewalls into the Curve Protocol.

Project Details:

Core Components

Documentation

  • Research results outlining the benefits and technicalities of implementing Firewall systems to Curve.
  • Detailed description of all functionalities.
  • Step-by-step guide for Firewall implementation.
  • Limitations and potential obstacles in the use of Firewall
  • Suitable governance solutions for handling a triggered Firewall event
  • Outline and analysis of Firewall event strategies
  • Github repository, complete with all relevant unit tests

Frontend (if needed)

  • Dashboard:
    • Security monitoring & analytics
    • UI Components, displaying relevant Data
    • UX improvements, seamlessly integrating new functionality to the existing interface (if given)
  • Dashboard, displaying subgraph data relevant to Curve stakeholders
  • Governance Interface
    • For handling proposals in emergency situations, such as Firewall events

Smart Contracts

  • Development and testing of smart contracts to facilitate Firewall functionalities.

Subgraphs

  • Creation of subgraphs to gather data relevant to Firewall functionalities and Curve stakeholders.

Scope of Work, Milestones, Grant Allocation

We propose to divide the grant into three phases, corresponding to the complexity and novelty of the project. We are asking for initial funds for Phase 1 to get us started focusing 100%. Upon successful completion of Phase 1, we intend to discuss further phases and the corresponding funding requirements with Curve. Therefore, Milestones 2&3 are yet subject to change, depending on the alignment with the curve team, after completion of Milestone 1.

Milestone 1: Research & Proof-of-Concept

  • Estimated Duration: 1 Month
  • FTE: 3 (*FTE = Full Time Equivalent)

This phase will focus on in-depth research and the creation of proof-of-concepts demonstrating how tailored firewall can enhance the security and functionality of Curve.

Number Deliverable Specification
1. Documentation We will provide both inline documentation of the code and examples that explain how on-chain security methods, mainly firewalls could be integrated to the Curve Protocol. All technical components needed to realise this, will be explained. The documentation will also include the detailed research findings that address → technical limitations, → potential risks, → benefits, → different technical and non-technical implications for Curve
2. Proof-of-Concepts Based on examples that result from the conducted research, functional prototypes will be build proving its technical feasibility. This proof-of-concepts also serve for demonstration purposes to the Curve and its stakeholders, so they have more material to evaluate the further future of this Grant.
3. Technical Blog Post A technical blogpost explaining the proof-of-concept and research findings

Milestone 2: Prototyping & Production (subject to change)

  • Estimated Duration: 2 Months
  • FTE: 3

The second phase will concentrate on the development and testing of Firewall prototypes based on the findings of the first phase. The prototypes will showcase the practicality and benefits of integrating on-chain firewalls into the Curve protocol.

Number Deliverable Specification
1. Smart Contracts Development and testing of smart contracts to facilitate Firewall functionalities.
2. Subgraph(s) Gather data relevant to Firewall functionalities and Curve stakeholders. → relevant for voters and governance overall
3. Frontend(if needed) → Dashboard: security monitoring & analytics, UI Components, displaying relevant Data, UX improvements, seamlessly integrating new functionality to the existing interface (if given) → Dashboard, displaying subgraph data relevant to Curve stakeholders → Governance Interface: for handling proposals in emergency situations, such as firewall events
4. Advanced Features → subject to discussion
Our team will also assist in marketing and explaining the integration, including:
  • Public blog post and co-marketing.
  • Detailed technical explanation of firewall architecture.
  • Detailed analysis of security guarantees.

Milestone 3: Testnet Deployment (subject to change)

  • Estimated Duration: 1 Month
  • FTE: 3

The final phase will involve the testnet deployment of all Firewall components, followed by rigorous testing and refinement to ensure the system is ready for mainnet audit and potential integration into Curve.

Number Deliverable Specification
1. Testing Guide All code will have proper unit-test coverage (e.g. 95%) to ensure functionality and robustness. In the guide we will describe how to run these tests
2. Public and clearly documented open-source repos for: → Smart Contracts, → Subgraph(s) → Frontend (if needed) We will provide both inline documentation of the code and a basic tutorial that explains how a user can (for example) spin up the application. Application is up, it will be possible to send test transactions that will show how the new functionality of Firewall & governance functionalities, work.
3. Testnet deployments all networks
4. Informational Publications → Blog Posts, etc.
5. Audits → Auditing the Core Infrastructure
Our team will also assist in preparing the system for mainnet deployment, including:
  • Coordination with Curve for all relevant security audits
  • Operating and maintaining relevant components such as the frontend and governance interface
  • Providing an open-source implementation for backup

Who we are

TurtleShell is a web3 public goods initiative designed to bolster the security of DeFi protocols by introducing on-chain Firewalls to detect malicious activity on protocols and prevent hacks.

We develop open-source security infrastructure tools, such as our Firewall SDK.

Team

Valerio Fichera

Philipp Keinberger

Vincent Daubry

Resources:


The TurtleShell team is fully committed to this project and excited about the potential to enhance the security, reliability, and functionality of Curve through the integration of Firewalls. We look forward to your consideration of this proposal.

link to notion page of this proposal

Do you believe this proposal is worth to be further considered?

  • Yes
  • No

0 voters

8 Likes

There are a lot of open questions, so I think starting with stage 01 and then moving forward makes sense.

Would Total liquidity be the only option?
This could also be triggered when there is no hack in case of extreme market conditions when the pools are needed the most.

In theory, good in practice, not so much (It seems).
Do you have proof that it works?

What would be the requested budget for phase 01?
Estimated budget for Phases 02 & 03?

Thank you.

1 Like

@ON-DeFi thank you for your feedback :slight_smile:

According to the estimated time and work efforts we propose the following budget for this grant project:

Milestone 1: Research & Proof-of-Concept

  • Estimated Duration: 1 Month
  • FTE: 3 (*FTE = Full Time Equivalent)
  • Costs: $15,000
  • Estimated delivery time: early September

Milestone 2: Prototyping & Production (subject to change)

  • Estimated Duration: 2 Months
  • FTE: 3
  • Costs: $30,000
  • Estimated delivery time: end October

Milestone 3: Testnet Deployment (subject to change)

  • Estimated Duration: 1 Month
  • FTE: 3
  • Costs: $15,000
  • Estimated delivery date: end November

*as calculation base we determined an equivalent of $5k as gross compensation for a contributor per month of full-time work commitment.
** Milestones 2&3 can be re-evaluated after reaching Milestone 1.

Total Liquidity is not the only option. Any quantitive metric can be used as parameter. You may choose to use one or multiple security parameter(s). Each security parameter is referenced by its creator (an address) and a unique identifier. These identifiers may be a token address, a hash, or any arbitrary Solidity bytes32 value. This means a computed formula consisting of multiple values would also work.

However, you are right. A poorly configured firewall can lead to false positives in cases where there are no hacks (e.g. whale exiting funds). This would result in a bad UX at worse, which is still more favourable than loss of funds, but not a final solution to this problem. To prevent that from happening, we have taught of a couple different measures, that yet need to be stress more tested. Besides defining the right parameter, a big role in this would also play the measures to be taken once the firewall is triggered.

Regarding having a proof-of-work, we have built a Demo where we mocked a Lending & Borrowing Market with modifications to make the contracts vulnerable. We then implemented a Firewall to prevent it from being hacked, by locking the funds and pausing the protocol once the firewall has been triggered. The Demo shows to versions, one with and one without the Firewall. The Demo can be found here.

Thank you for consideration :slight_smile:

1 Like

@valerio_eth

The reentrancy losses observed in the past week across the Curve ecosystem highlight the critical need for real-time detection and protection for deployed contracts and protocols.

We think your proposal is worth further consideration and we are keen to explore ways our team (Cube3.AI) can collaborate with your team in order do develop security tooling and improvements that could benefit the community at large. We have been doing a lot of work on "AI firewalls” I think there is an opportunity for us to work together.

We would like to support this initiative.

2 Likes