[DRAFT] SIP 0008 - Sovryn Bug Bounty Program on Immunefi

Bitocracy SIP | Sovryn Improvement Proposals
1

Hello Sovryn community!

I’m Travin Keith, Co-Founder and COO of Immunefi, a bug bounty platform focused around crypto aimed at preventing catastrophic hacks by setting up bug bounty programs that encourage white hat hackers to look through the code as well as economically incentivize black hat hackers to disclose vulnerabilities instead of exploiting them. Over the past few weeks, I’ve been in discussions with the Sovryn team, namely yago and light, about the creation of a bug bounty program for Sovryn. With their help, I’m happy to present to you our draft for SIP-0008 - [DRAFT] SIP-0008 - Sovryn Bug Bounty Program on Immunefi - Google Docs

We have two things we’d like feedback on from the community, highlighted in yellow on the document:

We were considering having the bonuses paid out in SOV in relation to the BTC value. We thought that this would be a great way to encourage very skilled white hat hackers to continue being a part of the Sovryn community. We could also have the bonus payout have a vesting period of a few months in this case for critical bugs to address market liquidity concerns, perhaps scaling down as well depending on the bonus applied.

We thought that the easter egg idea would be a nice fun way to encourage people to look through the code, but at the same time not feel like they wasted time if they didn’t find anything worth reporting. The rewards would be unique NFTs however, and not BTC or SOV. However, we also don’t want to flood the ecosystem with too many of these, so we wanted to hear from you on how many of these you think would be good.

We had some discussions about whether to have the bug bounty payouts pegged to BTC or a set USD amount, but given that the treasury is in BTC, we thought that this would be best since the request for funds to be allocated for the bug bounty payouts was quite large and attempting to forecast values even over a year would be too difficult. In any case, we feel that this is fine as the value of the Sovryn code increases as the value of BTC increases as well.

To clarify something with the Total Funds Requested section, we are only asking for a commitment of 6 months for our premium bug report triaging and management service because the current fee structure is experimental, and this might end up being a loss for us in the future. We understand the need to plan ahead though, so we are happy to commit to this for now, and will re-evaluate shortly before the 6 months are over and create another SIP, or continue on if there are still funds remaining that can cover our new rate.

Thank you everyone for your time. I look forward to hearing your thoughts!

7 Likes
2

Would you like the feedback here on the forum, or in the document itself?

3

Here would be better as changes will be applied to the document based on feedback, though I’ll make a post here as well whenever changes are made.

4

Hi everyone,

Thanks for everyone’s feedback so far. I really appreciate the time taken by all to give feedback to our SIP.

I’ve now updated the document to reflect recommendations from the community. Changes to the document are highlighted in green.

Here’s a quick rundown of the changes I made:

  • Clarified that the Sovryn team makes the final decision on the severity level
  • Added some conditions with regards to what happens if there are 3 or less bug reports in a given month (no fee for the month and total is carried over) as well as spam reports
  • Additional promotion on Gitcoin as desired by the community
  • Clarification that the total funds requested are not to be fully transferred over to the Immunefi team but rather just earmarked by the treasury
  • Clarification on the fee for bug report triaging and management being a consumable based on work completed.

Additionally, we’d still like some feedback on the bonus payout being in SOV or BTC as well as how many NFTs will be released for the Easter Egg component.

1 Like
5

image
image1155×737 58 KB

Just for reference, here are the top few bounty programs on Immunefi.

I think we should be in the top 3 if not top 2, as the ceiling for AUM will be several digits more than the proposed bounty allocation fo $250,000.

I would like to see Sovryn allocate between $750k and $1M.

1 Like
6

I think this is a great SIP and testament to the permissionless nature of Sovryn, that other groups and communities are excited to join the Sovryn project, collaborate and contribute.

To my mind, our approach should always be security first. Sovryn protocol is likely to facilitate transaction volume on an enormous scale. The more friendly “adversarial” attention the codebase attracts from talented hackers, the more confident we can all be in using it. Now, before the protocol is host to thousands or millions of bitcoins-worth of value is the time to stress test our system aggressively. I would support expediting this SIP, and getting the bounty going ASAP.

I think that the bonus structure is a fantastic way to attract attention. In fact, I would be supportive of having the maximum bounty be an eye-popping number (eg. $1m), all the better to attract attention to the bug-bounty.

I do think, that as is common, the maximum payout (before bonuses) should be limited to 10% of the current value at risk.

I have been impressed by the level of engagement this topic has attracted from the community (particularly in Sovryn Discord). It is our most controversial SIP yet, because the community clearly wants to be careful with funds and get the maximum bang for its buck. So I would encourage @TravinImmunefi to try and sweeten the deal :slight_smile: Let’s get this done!

2 Likes
7

Travin, you have won me over.

1 Like
8

Ingalandia, thanks for looking into this! Unfortunately though the sorting is not by allocation but rather by critical payout. In the current state though the 5.5 BTC for critical with the 25% bonus is now at around $360k.

Yago, thank you for your support! I’m happy that you see things that way as well. Having more of these specialists on Sovryn’s side is indeed something that would further generate confidence in the platform.

I agree as well about having an eye-popping number. I’ll take a look at the figures again tonight and have things re-structured since it seems others also want to be on the top, which is great! Having the limit to 10% of the value at risk though is certainly reasonable and I’ll make sure to keep that in there as well. I’ll also try to find other ways to further sweeten the deal for Sovryn :slight_smile:

Ponjinge, thank you! It means a lot to me and it also meant a lot to me how you communicated all your concerns and issues you had with this SIP in a very professional way, together with everyone else who joined in the discussion. Every concern you brought up along the way was reasonable and thoughtful and I appreciate greatly the time you took to do due diligence with us and explore other options.

1 Like
9

Hi everyone,

I was able to make some changes to the draft in order to account for the additional feedback provided. After discussing with @jonahimmunefi, we were also able to sweeten the deal a bit more for Sovryn. :slight_smile:

I’ve now removed the green highlight for all previous changes and had the new changes highlighted green for easy tracking on what’s new. As like before, here’s a changelog of the changes that were made, in the order of which they appear:

  • Increase the max payout of a critical bug to 18 BTC, though still with the 10% cap limit based on the funds at risk. This would put the reward for Sovryn close to the top of the list and will continue to push up as the price of BTC goes up. At the time of this post, with the bonus of the first week in place, it would be at the top of the Immunefi list.

  • I decided that it would be best to have the bonus be in BTC since that is the currency for the funds requested anyway. If ever the Sovryn team would deem it to be beneficial to add SOV on top of the bonus in order to foster the retention of the highly skilled whitehat hacker within the Sovryn community, we will leave it to their discretion.

  • A plan for what extra promotion Immunefi will do for the Sovryn bug bounty program, beyond the usual promotion for most clients, including one part from myself personally as a way of saying thank you to everyone from the Sovryn community who has reviewed this SIP and contributed to it moving forward :slight_smile:

  • Added further clarification that feature requests and best practices critiques are out-of-scope. This was already the case, but I thought that I would add it as it seems to have been something that bug reporters for web/app bugs missed.

  • Increase the requested amount from 24.2 BTC to 25 BTC, as an 18 BTC payout with the 25% bonus and the 10% Immunefi fee after that would result in 24.75 BTC, being over the limit of the initial requested amount.

  • The initial requested amount was meant to account for up to about 4 critical bug reports with full reward. However, with the increase of the maximum payout to 18 BTC, (about $1m) I needed to add a clause to replenish the allocated amount throughout the bonus period in order to prevent the bug bounty program from becoming a one-and-done program with a single max critical payout. In order to ensure existence after the bonus period as well, I am requesting a one-time replenishment to 25 BTC in order to be able to pay for one max critical payout and thus create continued security for the Sovryn platform from catastrophic hacks. In the event that more funds are needed, a separate proposal will be created.

  • Added some further information about our plans with regards to going beyond the 6 months of bug report triaging and management as well as in the event of the funds not being enough.

  • Added a section of Immunefi Accountability that includes monthly reporting while Immunefi is doing the triaging and management.

Still Needed

I still need some guidance on the number of NFTs that would be able to be created for the Easter Egg section. After all of these bonuses though as well as the increase to $1m, I’m starting to feel that the section might not really be needed. Plus adding in some easter eggs in the smart contract code (just on GitHub, not the deployed ones) might be too easy to find if not hidden well, and we would need the Sovryn team to do this for us. However, I’m happy to hear further thoughts about this so we can get this finalized and pushed through for voting.

1 Like
10

@TravinImmunefi I think I would like to see 2things:
1- I am concerned that $1000 per 10 bug reports my be high, and could easily get out of control
2- I’d like to see a countdown clock on the Immunefi webpage, promoting the bonus period.

2 Likes
11

Thanks @yago!

For #1, given its nature, we’ll probably be getting more web/app bug reports than smart contract ones. It seems that most of the concerns are with regards to the smart contract side of things, so what could be possible is removing the “Low” rewards for Web/App bugs. Then anything reported as Low or None would simply not be accepted and wouldn’t count as part of the program. Though bugs reported as Medium or higher but then downgraded to Low due to impact factors would still count towards the moderation count as time is spent validating the vulnerability, investigating the impact, and engaging with the bug reporter, having this would discourage people from spending time reporting bugs that have that level as they may get no reward. For quick reference, here’s the Severity System we have.

In addition to this, here’s some other things that we would do. Some of these though are already how we normally operate for the moderation service:

  • Group set up on Keybase (or Discord if preferred) for active communication regarding the bugs that are being submitted so we can quickly make adjustments as needed (e.g pausing web/app bug reports, having some reports just go directly to the Sovryn team, etc.)
  • Going above $2k (up to 20 bug reports) in any month requires approval from the Sovryn team on that channel to increase beyond that.

Please let me know if this sounds good, and I will make the necessary modifications. I also just realized that I forgot that on Discord it was discussed that there might be a problem of lag of Immunefi responding to bug reports. I will add in the subsequent edit to the draft that high and critical smart contract bug reports will be immediately sent to the Sovryn team before validation work begins by the Immunefi team.

For #2, this is not a problem at all. I also got confirmation that this can be done on the homepage itself.

12

I’m in favor of the proposal. I think, the bug triaging service would come very handy. Having you guys deal with user reported bugs first, would save us a lot of time and effort.

I also love the easter egg idea and will ask the solidity team to come up with some fun ideas. Maybe we can have an internal reward for the coolest easter egg ideas :wink:

You suggest to give away unique artwork NFTs on Sovryn as reward. With unique, do you mean each piece should be unique? So, let’s say 3 critical bugs are found, should each finder get an unique NFT? Or should there be just one Bug-Hero artwork, which can be linked to more than one NFT? Should the art be prepared beforehand, so we can show what the NFT looks like? Or should we just create it if/when needed?

Regarding the out of scope section. Not sure if that’s obvious and doesn’t need to be added, but i would also include issues which have already been reported and are therefore known to the team to the out of scope list, else we might end up paying multiple times for the same bug.

1 Like
13

Awesome SIP @TravinImmunefi

The concerns I have are the below:

  1. It says “Bug reports that are considered spam, e.g copyright date issues, are not counted”

A bit elaboration on that would be nice. As there could easily be many people who will try to get atleast a low level threat who might send some weird threats according to them.

Also, I believe it should also include as @Ororo said, the issue already reported by someone else.

  1. Another thing regarding the SIP is a bit of inconsistency on the Smart Contract Critical Issue Payment and High Issue Payment. One get’s paid 0.5BTC, while other is a staggering 18BTC (or 10% of the funds at risk) which is 36x of what High issues get. This is a subjective remark, not an objective one.

  2. The Bonus which we provide could be given through SOV rather than BTC (SOV equivalent to BTC or some other structure). Another subjective remark, and maybe Yago can chip in on this. I think it was SOV before but was changed for some reason.

  3. One more thing: “For the bug bounty component, we request the allocation of enough BTC to cover up to 4 validated critical bug reports at full value, plus the 10% fee for Immunefi, payable upon accepted bug reports. This comes to a total of BTC 25.”

Isn’t it 1 validated critical bug, rather than 4?

18BTC + upto 25% bonus + 10% Immunefi Fee = 25 BTC (Approx)

2 Likes
14

@powerhousefrank Thoughts on easter eggs? Overkill?

15

Loved the idea of having an easter egg, but not sure about the implementation in an Open Source project :sweat_smile:

16

Thanks again everyone for the thoughtful responses!

@Ororo

You suggest to give away unique artwork NFTs on Sovryn as reward. With unique, do you mean each piece should be unique? So, let’s say 3 critical bugs are found, should each finder get an unique NFT? Or should there be just one Bug-Hero artwork, which can be linked to more than one NFT? Should the art be prepared beforehand, so we can show what the NFT looks like? Or should we just create it if/when needed?

I think it would be great for it to be created if/when needed, perhaps specific to the bug bounty hunter themselves.

This is for Smart Contract/Blockchain bugs though. I will clarify this aspect.

Regarding the out of scope section. Not sure if that’s obvious and doesn’t need to be added, but i would also include issues which have already been reported and are therefore known to the team to the out of scope list, else we might end up paying multiple times for the same bug.

Ah, yes, we have a platform-wide rule of first-come-first-serve. I will clarify this on the document to remove any lack of clarification.

@powerhousefrank

  1. It says “Bug reports that are considered spam, e.g copyright date issues, are not counted”
    A bit elaboration on that would be nice. As there could easily be many people who will try to get atleast a low level threat who might send some weird threats according to them.

Sure, I’ll add more details to the document regarding what constitutes as spam. Low-level smart contract bug reports should be easy to see as spam or not spam, but for web/app it’s a bit more time consuming, hence my recommendation to have it removed.

Another thing regarding the SIP is a bit of inconsistency on the Smart Contract Critical Issue Payment and High Issue Payment. One get’s paid 0.5BTC, while other is a staggering 18BTC (or 10% of the funds at risk) which is 36x of what High issues get. This is a subjective remark, not an objective one.

The difference is mostly due to the impact potential of a critical smart contract bug vs a critical web/app bug. With a web/app bug the most critical one directly affecting user funds would be spoofing, but these tend to be surfaced fairly quickly as users would complain that deposits weren’t going to their account or withdrawals weren’t going to the right addresses. However, more importantly, it’s much more difficult to attribute the amount of funds at risk so putting the same restriction is not really enforceable and it’s easy to come up with different numbers. We’ve also found that about $25k is enough to encourage many web/app white hat hackers to look through the code. We’re of course happy to increase the amount if desired by the team, but our overall advice is that it isn’t needed.

The Bonus which we provide could be given through SOV rather than BTC (SOV equivalent to BTC or some other structure). Another subjective remark, and maybe Yago can chip in on this. I think it was SOV before but was changed for some reason.

I had it as TBD before. I think there would be benefits of having it as SOV as well, but one issue is that the amount of SOV requested would need to be quite a lot as the bonus factor is dependent on the BTC:SOV price ratio, which is currently unclear since it’s still in the very early days.

One more thing: “For the bug bounty component, we request the allocation of enough BTC to cover up to 4 validated critical bug reports at full value, plus the 10% fee for Immunefi, payable upon accepted bug reports. This comes to a total of BTC 25.”
Isn’t it 1 validated critical bug, rather than 4?

Ah, I forgot to update this part. It was changed to make it so that one is covered, but with a replenishment clause in place instead of asking for a large sum at once.

==

Regarding the easter egg part, perhaps I think I can just keep it in, but have it at the discretion of the team to do. I think it won’t need to be published on the actual live smart contract, but just on the GitHub itself. Also since there’s no direct financial cost to this other than time spent to make a worthwhile NFT, it doesn’t affect the rest of the proposal. Would this be ok?

17

Thanks. A few last things- as I hope we can vote tmrw:

  1. Easter Eggs - I don’t think we have the bandwith for it. Would remove.
  2. Payment in BTC or SOV. Up to 50% to be paid in SOV under teams discretion (potentially with some level of vesting)

Otherwise, i think we should vote and get going.

18

@TravinImmunefi please create a PR to GitHub - DistributedCollective/SIPS after the final changes. If you need help, ping the dev team on the discord.

19

Hi everyone,

I’ve now updated the document, again with changes highlighted in green, according to further feedback I’ve received here. Here’s a rundown of them in the order of which they appear:

  • Added clarification on what constitutes a bug report not included in the tally towards the bug report triaging and management service
  • Added the measures mentioned in this thread regarding keeping the costs in check.
  • Removal of the Low reward for Web/App bugs
  • Clarification that the bonus NFT reward is only for critical and high Smart Contract/Blockchain bugs
  • Added that the NFTs will be designed only if/when a reward is given.
  • Added that up to 50% of the bonus may be payable in SOV and possibly with a vesting schedule
  • Removal of the Easter Egg section
  • Added the countdown clock inclusion for both the bug bounty page and the Immunefi homepage
  • Added further clarification on the scope of the web/app assets
  • Clarified that in the event there are two or more bug reports about the same vulnerability, only the first one gets rewarded.
  • Corrected “4” to “1” from the previous modification. (was missed previously though the rest of the paragraph already adjusted.

Will do!

20

Pull request now created - Create SIP-0008.md by TravinImmunefi · Pull Request #7 · DistributedCollective/SIPS · GitHub