Akash Network Provider Review + Incentives Proposal

[ThanosNode]

Objective:

Increase adoption and trust within the Akash Network for GPU and compute deployments by implementing an interoperable provider review platform that augments provider incentives.

Solution:

A two-tier review system offering both user-based, and audit-verified data to inform deployment decisions. This can be used for basic due diligence, in addition to a more detailed diligence exercise to inform executive decisions for complex workload deployments. Similar to Amazon Reviews or AirBnb reviews, signed reviews and associated information can be shared beyond the Akash microcosm. Adjacent ecosystems, web2 entrants, and interested parties with a vested interest (or those who wish to participate in a limited capacity) may utilize the reviews to educate their constituents.

Furthermore, should the need to utilize referral links arise, the review system will be a launching pad for a referral program with live signed reviews, similar to DigitalOcean or Amazon referral programs (though referrals are not part of this proposal iteration).

Tier 1: User Reviews:

Basic rating: 1-5 star system for overall provider experience.
Descriptive feedback: Users can share details about deployment performance, reliability, communication, and support received.
Provider information: Includes experience level, hardware configuration, and recommended use cases.
Recommendation system: Users can choose to recommend the provider or not.

Tier 2: Audit Integration:

API integration: Connect with a trusted audit service to pull configuration data on provider resources (e.g., security, compliance).
Verified information: Display audit-backed details about resource specifications, uptime, and security measures.
Increased transparency: Users can make informed decisions based on both subjective and objective data.

Incentives for Providers:

User & Deployment retention: This will extend deployments by prompting reviews whenever a user is closing a deployment.
Incentivizing Deployers: 30-day +/- deployment extension to review their deployment experience (30 days or more deployments).
Provider incentives Pool: Incentives may tap into the providers incentives pool (Prop 448) to pay for review incentives
AKT token rewards: Based on positive review scores and audit verification results.
Increased visibility: Providers with high ratings and active reviews appear prominently in search results.
Direct user referrals: API allows third-party platforms to display reviews and incentivize deployments through referral links.

Open Source Development: The entire review application will be open-sourced and accessible to partner, third parties in and outside the Akash and web3 community for contribution and more.
On-chain deployment: The application will be deployed directly on the Akash Network for decentralized governance and transparency.

Estimated Development Cost:

Development hours: 200-250 hours (approximately 4-8 weeks + 2 week contingency)
Milestone / Deliverables break down in 2-week sprints.
Cost: $55,000 (assuming an hourly rate of $200-$250 for developers)

Benefits of Modularization:

Increased Accessibility: By decoupling the functionality from the full application, we significantly lower the barrier to entry for new users. A self-contained component makes it easier for diverse audiences, with varying needs and preferences, to adopt and utilize the core functionality.
Enhanced Scalability: Modularization simplifies deployment and facilitates independent scaling. This provides greater flexibility in adapting the component to different use cases and user volumes, without impacting the broader platform.
Improved Adoption: Offering a standalone solution allows organizations to integrate the desired functionality into their existing workflows and platforms with greater ease. This reduces friction and encourages wider adoption across diverse user groups.

Strategic Advantage:

Separating this functionality aligns with our goal of promoting modularity and agility within the provider platform. This approach fosters:

Faster Innovation: Independent development cycles for the standalone component enable quicker iteration and feature updates, allowing us to respond to user needs and market trends more efficiently.
Reduced Complexity: A modular architecture simplifies platform maintenance and upgrades, leading to improved operational efficiency and lower overhead costs.
Stronger Ecosystem: By offering a distinct and well-defined component, we attract a wider range of developers and potential partners, fostering a thriving ecosystem around the platform.

Benefits:

Increased user trust: Improved transparency and provider accountability lead to more confident deployments.
Boost provider competition: Incentives encourage providers to improve offerings and services.
Accelerate network growth: Enhanced user experience attracts new deployments and drives adoption.
Open and accountable development: Community-driven and on-chain governance ensures sustainable growth.

Next Steps:

Community discussion and feedback: Open forum to discuss proposal details and potential improvements.
Development team: Resources have been identified and recruited across the current ecosystem including incumbents and new entrants.
Phased development: Prioritize core functionalities while allowing for iterative improvements based on community feedback.

Call to Action:

We encourage the Akash community to discuss this proposal and consider its potential impact on network growth and user experience. With your support, we can build a robust review system that incentivizes providers, empowers users, and ultimately strengthens the Akash Network.

NOTE:

This is a proposal and the specific details, including funding allocation, development timeline, and technical implementation may be subject to further discussion and refinement based on community feedback. Additionally contributions are welcome, and contributors may be part of the effort.

The estimated timeline to delivering this capability is May, 15th, 2024.

Break down of costs / milestones, and deliverables are below;

Backend Work breakdown - 3 Phases

Preparation and Planning Week 1: 03/1/2024
Objective: Finalize project scope and technical specifications and gather all necessary resources.
Deliverables:

Technical specifications for the review system.

System Architecture and API Design
Objective: design the system architecture and API endpoints for user reviews and audit data integration.
Deliverables:

1. System architecture diagram.
2. API design document for both user review submissions and audit data retrieval.

Database Design and Backend Setup

Objective: Develop the database schema and set up the backend environment.
Deliverables:

1. Database schema for storing user reviews, provider information, and audit data.
2. Initial backend server setup and configuration.

API Development and Integration - Part 1 (week 2-4)

Objective: Develop APIs for user review submissions and basic provider information retrieval.
Deliverables:

1. Working APIs for posting and retrieving user reviews.
2. Basic provider information API.

API Development and Integration - Part 2 (week 2-4)

Objective: Integrate with audit services to pull verified provider data and implement the recommendation system.
Deliverables:

1. Audit data integration API.
2. Recommendation system logic and API.

Testing and Feedback Integration (Week 4-5)

Objective: Conduct thorough testing of all components and integrate feedback from initial user testing.
Deliverables:

1. Testing report, including bug fixes and performance improvements.
2. Updated API and system components based on feedback.

Frontend Integration and Final Testing (Week 6 + 1 week contingency)

Objective: Finalize frontend integration, conduct comprehensive system testing, and prepare for launch.
Deliverables:

1. Fully integrated review system ready for deployment.
2. Final testing report, including load testing and security audits.

User documentation and developer guides.

Frontend & Integration Handling

Summary:

Tasks outline related to provider review dedicated frontend, documentation & review-modal integration handling onto existing tools like cloudmos.

Tasks:

UX research & UI Design (2 weeks)

1. Conduct UX research for dedicated frontend, documentation & integration feedback modal onto cloudmos.
2. Plan & Design the UI for dedicated frontend and integration feedback-modal onto cloudmos.
3. Gather feedback on the final components of design and feedback modal.
4. Follow strict branding guidelines & standard tailwind components.

Development & Integration (3-4 weeks)

1. Develop the Dedicated UI interface.
2. Define proper components and proper API request handling to fetch and display data.
3. Integrate feedback modal onto cloudmos, that pop-up as soon as the deployer closes the deployment.
4. Define API documentation , can be used by others to fetch and display review data anywhere.

Testing, Validation & error handling (1-2 weeks)

1. Test the FE configuration and error logs.
2. Identify and address any errors or misconfigurations in the backend by generating a report and implementing filters.
3. Conduct a comprehensive test of the solution's behavior and generate a functionality report to identify and address any significant bugs.
4. Hand-off!

[rodri-r]

Hey Alani

I do believe that this is a good idea and that having a user review/rating for providers and auditing is necessary, but you may not be aware of this https://github.com/orgs/akash-network/discussions/406 which already passed and I am almost certain that @piber-dev said they would be adding this to their new auditing service.

[ThanosNode]

Hi Rodri,
I reviewed discussion 406. This proposal is to have it modularized as a standalone component of the overall provider platform. By having it this way, we lower the barrier to utilization, scaling, and adoption by a wide variety of audiences. This simplification is further enabled by not roping it into a much larger application.
I believe modularizing the discussed functionality as a standalone component offers significant strategic advantages, enhancing accessibility, scalability, and adoption for diverse audiences. This aligns with our long-term vision for a flexible and robust provider platform that empowers innovation and drives user success.
Hope this clarifies the strategic positioning.
Thanks again for the input.

[piber-dev]

hello @ThanosNode, i want to reply as one of the team members developing the tier-based auditing system. the tier 1 (user reviews) is something not planned for the proposal, however we may develop it after the proposal deliverables on our end without requesting more funding for this. this is something both we believe in and something that some SIG recently discussed IIRC.

furthermore, we are not planning to touch some things related to tier 2 such as security/compliance and there is definitely collaboration and development to be made here. other parts seem redundant here, you might want to take a second look on what we are planning to do and see if you will be able to recycle or integrate parts of our tools.

ps: regarding the modularity, we are planning to integrate some services to the cloudmos indexer if possible however we want to make it possible for anyone to spin up their own copy of our tools and services.

[ThanosNode]

Hi @piber-dev I have reviewed your proposal, and you are on the right part. I believe these are two separate exercises strategic vs. tactical. The scope of your proposal is broad, and verticalized in-that it addresses core areas that the network needs today and focused on providers. Our proposal is horizontal, in-that is it focused on demand generation and market sustainability.
In a two-sided marketplace, it's strategically cumbersome to lump demand and supply side into a single iteration.
Our proposal focuses on the demand side...i.e. deployers.
Hope that helps with clarification. Thanks again for the input, very much appreciated.

[shimpa1]

Hi Alani,
I like the idea of a user-driven feedback system. It is a great idea. I do have a few questions;

1. This system works as long as the user feedback is actually legitimate however it can easily be used as an anti-competition tool. There has to be a system to revoke good or bad feedback if it's malicious. I haven't seen any such moderation tool being planned as a deliverable in your proposal and I think without it, the system can do more harm than good. If you do agree with me how do you plan on moderating the feedback?
2. The developer hourly rate seems extreme.
3. I will echo what others have already asked; there's already a project going on that is very similar to yours and a lot less expensive. Do you believe there is some feature that you will be adding to justify the cost of developing two systems in parallel?

thank you!

Shimpa

[ThanosNode]

Hi @shimpa1 , good to hear from you.
Reviews are signed under one of two conditions;

1. Existing deployer who has either had a deployment older than 30 days (or 15). They will be prompted either at the 15 or 30 day mark, OR at the point where they are closing their deployment if shorter than the established time period (15 or 30 days). To prevent abuse and malicious behavior, a deployment less than 24 hours will cost AKT to post.
2. No deployment. If an entity just wants to post a review and the have had no deployment, they it will cost $AKT (beyond just signing) to post a review with no associated deployment. This could be 2 - 5 AKT depending on what the general consensus on what the amount will be. This amount can be amended via the admin dashboard that will be used to manage the process.
3. The hourly rate is actually standard given assembling a team, the short timeline, and delivering this in a focused manner with undivided attention. We have to be careful with quality management to ensure a reasonable balance between resource allocation and timely delivery. In this case it's an all-hands effort with multiple teams involved, with contingencies baked into the estimate after consultations with some incumbents and new entrants. Additionally, this is not just "developer" time, this includes QA, UAT, Documentation, and Testing.....all beyond the familiar microcosm of the current Akash ecosystem. In other words, there will be people outside of our familiar space involved in this effort.

Finally, on moderation, signed reviews intrinsically has self governance as the system itself is self sovereign by having these barriers to abuse. Of course, should a person feel passionate enough to spend $AKT to post phantom reviews, they are backing it up with funds which may give credence to why they feel they need to back up their reviews monetarily.
We must avoid having a "moderation Tzar or entity" managing who gets to post reviews and who doesn't. By including the self moderation into the core design as aforementioned, this rectifies the issue of moderation.

Hope this helps.

Alani

[rodri-r]

Hey Alani, like I said before, I do like the idea of a review platform based on the "demand side...i.e. deployers", but like it has been mentioned before, there is already a project that is very similar and I also believe you are requesting too much for such a platform, which I am certain that a single developer could create for a lot less.

What I totally don't agree with you on, is your second point. Having an option for someone to leave a review without anyone actually knowing if they have used a provider or not or even ever used Akash, makes no sense to me and it could affect some providers and the network in many ways.

[ThanosNode]

Hi @rodri-r The incumbency of over-solving for providers, and the comfort zone on that side of the marketplace is reflected in your argument. This precisely juxtaposes the need to dislodge both sides of the marketplace in a single iterative effort. This has been attempted in the past with success on the provider side, but 4th quadrant results on the demand side. While I understand your position, I urge you to re-read the proposal in it's entirety as against focusing on a tactical component.

Furthermore, this isn't something a single developer can simply put together, if you understand the scope of the proposal in it's entirety. I would recommend re-reading the proposal in it's entirety to get a broader understanding of the scope, in addition to follow-on commentary.

This is why we are having this discussion, to elicit forward-constructive input. Not having used a provider and leaving a review could span multiple avenues as part of a due diligence effort, while costing $AKT on the part of the reviewer.

Alani

[cloud-j-luna]

What I totally don't agree with you on, is your second point. Having an option for someone to leave a review without anyone actually knowing if they have used a provider or not or even ever used Akash, makes no sense to me and it could affect some providers and the network in many ways.

I haven't yet gone through the proposal and I hope to have some time off work to review this. as I skimmed through the comments I do believe this concern can be solved by having the rating system on-chain making it easily verifiable.

As I said I'll add my thoughts on this proposal later, thank you! 😄

[shimpa1]

Alright,
I will repeat once more; there already is a community-funded program that does what you are suggesting to do. I see no point in pursuing two of them.
The authority that would have the moderation privileges needs not be all-powerful, only to have the ability to review the feedback and in special cases delete them. The modification can even be public.
Further, regarding the rates, the community is already funding a number of projects that don't charge nearly as much. Also, we are not in a hurry and would actually prefer to support someone (or an entity) that is already participating in community projects in one way or another.
All in all - although the idea is great, it is already being addressed.
thank you!

[rodri-r]

I think that with a little tweaking, we could use this https://github.com/astuto/astuto and adjust it to create this demand side/deployers review platform and it looks like it could easily be deployed to Akash. No need to reinvent the wheel or get a team of developers for this.

[ThanosNode]

Can we get additional contributors to this discussion?

[heyhewi]

Hey Alani,

Thank you for putting this up and speaking. Admittedly, I read it when you posted but didn't keep up with discussions. After listening in the Steering Committee and reading some discussion, are you able to break down costs a little further to add clarity to the proposal?

As a non-dev and not knowing what is reasonable, it would be helpful to at least see rough dollars/sprints tied to deliverables/milestones you hope to achieve. This could also help address any duplication of effort concerns. I like the high-level vision and have no doubts on your ability to perform, I'm just not fully understanding what success will look like.

Estimated Development Cost:

Development hours: 200-250 hours (approximately 4-8 weeks + 2 week contingency)
Milestone / Deliverables break down in 2-week sprints.
Cost: $55,000 (assuming an hourly rate of $200-$250 for developers)

[ThanosNode]

Hi @heyhewi Good input. Success encapsulates the below bullet points (milestone on the cost to follow soon);

Reviews are critical to help build trust on Akash — they’re an important way for deployers to deliver feedback, and ratings that help the community (and future deployers) make informed decisions, and understand what to expect when considering whether or not to utilize the network.

It not only helps to increase the credibility and reputation of a provider. Positive reviews can help to attract more deployments, which can lead to higher occupancy rates and more revenue for providers + token spend of course.

Incentivized reviews is also a strategic method to help prolong deployments and sustained provider utilization, thereby reducing churn ... .or “drive-by” deployments.

A fair review system is one that respects and protects the community’s genuine feedback, in addition to safeguards in place to help build trust in the review system.

Success:

Informed Decision Making - Provider reviews provide valuable information for potential deployers, enabling them to make informed decisions about their deployment plans, configuration etc. Reviews can reveal crucial details about the provider, their configuration, uptime, and the overall experience. This will quickly inform both operational and executive decisions.

Establishing Trust and Credibility - Deployer reviews are an essential tool for building trust and credibility between providers and potential guests. A provider with numerous positive reviews signals to potential customers that the provider is reliable and their node meets or exceeds expectations.

Higher occupancy rates: Positive reviews will lead to more deployments, which can help to increase occupancy rates and revenue. What's good for providers, is generally good for the network. If higher occupancy rates mean 75% or higher (shooting for 95%) have active deployments, that is a success criteria we should drive for via higher occupancy rates.

Improved network ranking: Good reviews can help to improve provider ranking when potential deployers search and sort providers by their preferred criteria.

Increased demand: Good reviews can lead to more interest in a provider(s). The ability for a deployer to sort providers by ratings criteria (GPU, Uptime, Review Rating, etc) will only increase conversion, deployments, and upward price elasticity. This is co-terminus with the proceeding point.

Avoiding Unpleasant Surprises - Negative reviews can be just as informative as positive ones, often highlighting potential issues or red flags that may be deal-breakers for some deployers. By reading deployment reviews, customers can avoid unpleasant surprises and select a provider that best suits their needs and preferences. Data points from both positive and negative reviews will inform network performance, provider, hardware and deployment duration insights. For example, gaming deployments on A100's over a 90-day period performed better and resulted in $AKT spend of X amount compared to V100's for medical imaging (one of many examples).

Constructive Feedback - Provider reviews provide valuable feedback, allowing them to improve their infrastructure and address any issues that deployers face, especially since there is generally** no direct communication link between providers and deployers. Negative reviews can be an opportunity for growth if providers take the criticism seriously and make necessary changes.

**“SuperProvider” Status - Positive provider reviews contribute to achieving a coveted “SuperProvider” status on the Akash network. Since the highest rated providers (sorted as part of due diligence effort/criteria determined by the deployer) will likely get the most deployments, this imbibes a culture of continuous improvement efforts based on feedback to providers.

Finally, the feedback plays a crucial role as a powerful marketing asset that holds significant influence over business relationships in the case of more complex workloads, GPU’s etc. Utilizing positive reviews across diverse platforms such as social media, email campaigns, and featured listings can effectively showcase reputable providers catering to distinct customer needs.
These reviews are also sharable beyond the native environment, also in PDF format (PDF report/snapshot 1 - 2 pager with config, rating information), in addition to other platforms including social media.
Incorporating compelling case studies and highlighting successful engagements serves as an excellent strategy to attract fresh users and enhance deployment rates.

Hope this expatiates for clarity.

Alani

[heyhewi]

Thanks @ThanosNode for the follow up and adding some more points. Again, I like the high-level vision and see value to reviews, looking forward to seeing the breakdown of costs/milestones/deliverables.

[ThanosNode]

@heyhewi Thanks for the follow up. Milestone breakdown will follow once resource estimates are calibrated and locked-in. Shouldn't take long.

[ThanosNode]

@heyhewi

Backend Work breakdown - 3 Phases

Preparation and Planning Week 1: 03/1/2024
Objective: Finalize project scope and technical specifications and gather all necessary resources.
Deliverables:

Technical specifications for the review system.

System Architecture and API Design
Objective: design the system architecture and API endpoints for user reviews and audit data integration.
Deliverables:

1. System architecture diagram.
2. API design document for both user review submissions and audit data retrieval.

Database Design and Backend Setup

Objective: Develop the database schema and set up the backend environment.
Deliverables:

1. Database schema for storing user reviews, provider information, and audit data.
2. Initial backend server setup and configuration.

API Development and Integration - Part 1 (week 2-4)

Objective: Develop APIs for user review submissions and basic provider information retrieval.
Deliverables:

1. Working APIs for posting and retrieving user reviews.
2. Basic provider information API.

API Development and Integration - Part 2 (week 2-4)

Objective: Integrate with audit services to pull verified provider data and implement the recommendation system.
Deliverables:

1. Audit data integration API.
2. Recommendation system logic and API.

Testing and Feedback Integration (Week 4-5)

Objective: Conduct thorough testing of all components and integrate feedback from initial user testing.
Deliverables:

1. Testing report, including bug fixes and performance improvements.
2. Updated API and system components based on feedback.

Frontend Integration and Final Testing (Week 6 + 1 week contingency)

Objective: Finalize frontend integration, conduct comprehensive system testing, and prepare for launch.
Deliverables:

1. Fully integrated review system ready for deployment.
2. Final testing report, including load testing and security audits.

User documentation and developer guides.

Frontend & Integration Handling

Summary:

Tasks outline related to provider review dedicated frontend, documentation & review-modal integration handling onto existing tools like cloudmos.

Tasks:

UX research & UI Design (2 weeks)

1. Conduct UX research for dedicated frontend, documentation & integration feedback modal onto cloudmos.
2. Plan & Design the UI for dedicated frontend and integration feedback-modal onto cloudmos.
3. Gather feedback on the final components of design and feedback modal.
4. Follow strict branding guidelines & standard tailwind components.

Development & Integration (3-4 weeks)

1. Develop the Dedicated UI interface.
2. Define proper components and proper API request handling to fetch and display data.
3. Integrate feedback modal onto cloudmos, that pop-up as soon as the deployer closes the deployment.
4. Define API documentation , can be used by others to fetch and display review data anywhere.

Testing, Validation & error handling (1-2 weeks)

1. Test the FE configuration and error logs.
2. Identify and address any errors or misconfigurations in the backend by generating a report and implementing filters.
3. Conduct a comprehensive test of the solution's behavior and generate a functionality report to identify and address any significant bugs.
4. Hand-off!