Help! Unscalable Airtable/GSheet records/row limit, any solutions please?

Hi, after spending 1mo. and hours on Soft frontend and Airtable backend, just saw that:

  1. Airtable has unscaleable 250k records/base limit (and that’s on the unquoted, highest Enterprise plan where you have to schedule w/ sales to get a quote) and

  2. GSheets has unscaleable 357k rows/sheet limit (even if GSheet integration worked right now and I only had 1 field column per record/sheet instead of actual 28 field columns, that’d still only be 1M records/rows per sheet).

This makes Softr not a viable/startup-friendly option as it’d not only require double the work of re-doing the entire frontend and “backend” upon the 250k-358k records limit, as 1) I can’t be sending/splitting user registrations between multiple sheets/bases, and 2) the whole use-case of my app is being able to search/filter the user directory, but it might likely mean losing all of the user data, because no way to tell how much of it is manually transferrable/migrate-able to any scaleable database solution. This also makes the “unlimited records per dynamic block” Softr enterprise feature seem like false advertising.

Really don’t want to call the past month’s of work a loss and have to start from 0 all over again, is there any potential AWS/other database/integration/custom code that I can use for backend instead with Softr, or do I really have to search alternatives/hire someone to code from scratch? Many thanks!

Hello!

Wouldn’t most of your tech stack will require enterprise plans when you reach enterprise levels of scale?

If Airtable costs $20/user per 50k rows, then 5 million rows could presumably cost $2,000/month.

Typically, the enterprise plans scale at a cheaper cost per unit than the quoted public pricing.

Hi, again, their max limit on the highest enterprise plan is 250k records/rows per base. I can’t split my user database between multiples bases/sheets, that’s illogical from a setting up/splitting standpoint (manually monitoring when limit reached, editing Airtable connection in Softr Users Panel + remapping 5 fields there + user registration form’s 18 fields in time before any lost users) and impossible from display/search/filter standpoint (I can’t show/search multiple bases as “one user directory,” nor should such be an “option”), any other ideas appreciated, many thanks!

Hi,
This is something that was discussed a lot with other members.

First question is: What is the state of your business? Early stage? Proof of concept? MVP?

Second question is: what is your current number of records?

Possible answers:

  1. You think of scalability and you are far away from it. Here, you’re asking yourself the wrong question and iterate is not part of your plan (and this is going to be an issue at some point, as a business needs to grow step by step)
    Solution: Softr will handle REST API this year. This would let you use other database (and real backends) like Xano, Supabase or others.

  2. If your current business uses a lot of records, (some businesses, even early stage, need a huge amount of records, but that is very rare).
    There, different options: you have to handle some record deletions (that is part of managing a tech stack, full code or no-code) periodically and/or re-design your database structure, which may not be adapted to what you do.
    Other possibility is to succeed to plug a self-hosted backend in Softr, which requires strong tech skills. I think some members, months ago talked about it, but I’m not sure, I can’t find the thread.
    Other solution: How we used Postgres with Softr to help our business teams

1 Like

Hi @matthieu_chateau and all,

I understand REST API and MySQL/Postgres are on the roadmap - are there any definite plans for release dates?

Our use case is a cold outreach platform, where the volume of prospects / client is c. 2500 per month, we quickly hit the volume limits and are managing this off-platform today. We either need to know when Softr can plug into a data source that can handle this or start to look at other no-code platforms - we’ve invested a lot into getting this far with Softr and really don’t want to change if we can avoid it.

Cheers!

Mark

Hi Mark,

I don’t have any info on any clear timeline.

What is your current database with Softr? Airtable or Gsheets?

Hi @matthieu_chateau,

We’re currently using Airtable.

Only 1 of our tables will ever get anywhere near to exceeding the limits -

We looking at spliting out the tables to 1 per customer and “hard-coding” 1 user group per customer in Softr Studio, duplicating the same table block multiple times, with each block only visible to a single customer user group looking at a different table each time to get around the limit as a short-term fix.

We could sustain this for a short while as we have only have around 15-20 customers currently but we have nailed our MVP and know how to acquire new customers so are moving into the phase where we looking to grow and this would become quickly unsustainable.

So before we put all of our eggs in the Softr basket I was looking for some assurance that either REST API or MySQL / Postgres is close? Otherwise we will need to consider Bubble or Retool or another Softr alternative.

Hi again - is there anyone else from Softr able to comment on how far off the “Under Consideration” data sources are on the roadmap? @artur?

I’m not looking for something written in stone, just some level of confirmation that they definitely will come, and roughly when … Q1 2024 for example?

Hi :wave: Mark,

I expect to have SQL support Q1/Q2 next year

—Artur

1 Like

hi @artur , any update on this or Xano? returned a year later and disappointed to not visibly see solutions to this + other unresolved bottlenecks (see post history). being VC-backed, you should understand having industry customers rewrite arch/migrate systems later is a non-starter vs setting up properly the first time, thx

@conf, we expect SQL support Q1/Q2 this year; XANO is on the list, too, but we haven’t got a clear timeline yet…

1 Like

@artur I would be happy to participate in early access testing for using Postgres as a backend for Softr, if such a program is planned. Thanks

2 Likes

same here @artur! This would make things so nice for me. (PostgreSQL)

1 Like

@dbsi and @aar0n noted will post once ready

1 Like

I could see it being useful for some components of within my site. For example, I need to create the SEO and Slug fields and airtable makes that part easy. It wouldn’t be easy in Postgres. On the other hand, if I need to sync records to a chart, it would be nice if it can pull from postgres.

Hey @artur the SQL support would be great to scale our projects too. I launched a research reporting tool with Softr and invested lot of time to build it up, but now I reached my limits with Airtable. I look forward Q2 (Q1 is gone ah ah) but have a couple questions for SQL integration:

  1. Will be a limit in the number of columns of data source? Because Airtable did it to 128 columns.
  2. Will be a limitation in the number of chart blocks per page in Softr? Currently is very short to just 5 dyamic blocks, but working with SQL makes sense to increase this capability.
  3. Will be an early access testing?

Thanks in advance for your feedback and congrats for this great job you are doing!

2 Likes