Ways to increase performance of a Softr website (and underlying database)?

[thijs]

Dear community,

In this forum I read that enough people (me included) experience long waiting times for Softr (and underlying databases) to have data processed and relevant blocks reloaded. For example: when an user clicks on a button with a ‘one-click-update’, a programmed delay of at least 5 seconds is needed to ensure proper reloading of list-blocks. I would rather see the website respond within 1-2 seconds, so therefore my question:

Are there tips and tricks on how to (further) optimize a Softr website (and perhaps also underlying database)?

To give you an impression of the page that is slowest to (re)load on my website: it has 18 blocks. 8 of these blocks are list-blocks that call upon 4 tables in Airtable (with use of multiple conditional filters). The amount of loaded data is very small (approximately 20 records, filled with mostly text and each record/tile has an image with a file size less than 200 kb).

1. I suspect that (communication with) the underlying database is the biggest culprit. One question I have is if another database may be faster to respond (such as Smartsuite) ?

2. What can be done to optimize processing time/responsiveness within Softr?
   For example: are 4 list-blocks with different conditional filters slower than 1 list-block (calling on the same table in the same database)?

Any help to shave off some (re)loading time is welcome!

[thijs]

Dear community,

I believe that I found an important driver/culprit of massive loading times: the use of lean/filtered views from your data source. This helped to improve the average loading time (of the worst part) of my website by a respectful 25% (from roughly 10 seconds down to 8 seconds).

Note that this 25% is based on my amateur tests on the most complex page of my website. This page consists of 2 list-detail blocks (no filters) and 10 list-blocks (each using between 3-6 filters).
The test was the total time to (re)loaded the same page 6 times in a row (to minimize any cut-off errors) before versus after improvement. This test I repeated for four different instances (list details block with different record id’s) of the same page.
This test resulted in total loading times varying from BEFORE: 56-62 seconds versus AFTER: 44-50 seconds.

So what was the improvement?
In my prior set-up I ran Softr list-blocks on unfiltered table views (grid views) from Airtable and applied multiple filters in Softr. In the new set-up I run Softr list-blocks on tailored table views from Airtable (that only show the required columns and rows for each Softr list-block). The only filters that remain in Softr are dynamic filters (related to user id and record id).

[artur]

@thijs thanks for sharing.

Few questions and comments:

• Are those pages public or gated for logged in users ?
• Sometimes the first request can be slower and then subsequent page loads should be faster, this is true for Airtable API as well as for our own caches to warm up and hold some metadata in the cache. Do you see slowness in all requests first vs 2nd, 3rd etc…
• How do you count page load ? waiting until all blocks are loaded ?
• Is your data frequently updated data ? or can it be cached for 10-30 mins ?

[thijs]

Dear community,

When I created my fully user tailored dashboard, the loading time was a massive 20 seconds (it had roughly 20 list blocks divided over 5 tabs to hide/show a few of them at the same time). To make these tabs I used the custom-code (to hide/show blocks) as explained here: New - unofficial - feature: TABS - Hide/Show multiple blocks horizontally - Guides & Help Docs - Softr Community

However, the use of tabs with the current Softr has two problems:

1. the order in which the blocks load differ from the order in which they’re placed in Softr - so from an user experience it can take the full 20 seconds before the first tab is loaded (much better would be if the first tab was loaded first and the rest thereafter in the background);
2. the time it takes to load each block is simply too much.

So as a solution to this:
1. Question to Softr (or the community): is there a possiblity to assign a loading-order to blocks?

2. Instead of using tabs, just make it look like you’re using tabs, but in reality navigate to different pages. This improved the loading time from 20 seconds down to 2-4 seconds of most pages and one page takes 7 seconds. This can be done with this piece of custom-code (copy/paste this custom code to each page, and for each page all you need to do is change the “button active”):

<!DOCTYPE html>
<html>
<head>
    <style>
        @import url('https://fonts.googleapis.com/css2?family=Inter&display=swap');
         .button-group {
            display: flex;
            justify-content: center;
        }

        .button {
            background-color: #000000;
            color: white;
            font-family: 'julius sans one';
            font-size: 14px; 
            font-weight: bold;
            padding: 2px 10px; 
            margin-right: 6px;
            padding-top: 4px;
            padding-bottom: 4px;
            border-radius: 4px;
            border: none;
            cursor: pointer;
        }

        .button:active, .active {
            background-color: #C5C9FF;
            color: #000000;
            outline: none;
        }
    </style>
</head>
<body>
    <div class="button-group">
        <button class="button active" onclick="window.location='/'">Topic1 (home)</button>
        <button class="button" onclick="window.location='/page1'">Topic2</button>
        <button class="button" onclick="window.location='/page2'">Topic3</button>
        <button class="button" onclick="window.location='/page3'">Topic4</button>
        <button class="button" onclick="window.location='/page4'">Topic5</button>
    </div>
    
         <script>
    const buttons = document.querySelectorAll('.button');
        buttons.forEach(button => {
        button.addEventListener('click', function() {
            buttons.forEach(b => b.classList.remove('active'));
            this.classList.add('active');
        });
    });
</script>
</body>

</html>

[thijs]

Hi @artur ,

Answer to your questions:

1. The (single) page is gated for logged in users
2. I just did a test of 8 individual reloads of the same website, this is the result in the order of testing - I don’t see much improvement of ‘warmed up’ caches (and the irony is that my users are supposed to look at each page ONCE - so this cache warm up won’t be of much benefit to my use case):

old version:
test 1: 10.4
test 2: 10.1
test 3: 9.3
test 4: 10.6
test 5: 8.8
test 6: 10.3
test 7: 10.3
test 8: 9.9

improved version (in seconds):
test 1: 8.6
test 2: 9.3
test 3: 7.5
test 4: 8.5
test 5: 6.8
test 6: 6.4
test 7: 7.6
test 8: 8.5

3. The way I counted a page load is like you say: I wait for all blocks to be visibly loaded (and in my first test I immediately hit F5/refresh to get to a continuous (re)loading of 6 pages in a row).

4. It depends on what you mean with frequent updating? I have an online configurator where users can (interactively select) whatever they like (to build a new house). The dataset has +300 records that are static (they remain the same). But to give the user a sense of overview, these records are divided into 40 menu-groups… so an user has to (re)load the same page 40 times to get though all records.

Hope this answers your questions. If not, I’m happy to further clarify.

[thijs]

Dear community,

And some more possibilities for (big) improvement of both loading times AND user experience are:

1. Allow Softr list blocks to connect to more than just 1 table. Conditional filters can easily help to filter/present the right data.

2. As long as point 1 is not resolved, opting for a less efficient database design perhaps makes Softr’s life easier loading (and filtering) this data. By less efficient I mean more rows that can be read by Softr as different records (instead of columns that can’t be interpreted/presented by Softr as records). I suspect that the sum of a lesser efficient database design combined with easier loading by Softr may overall improve the performance of my website. I will test this later (and update this thread).

Example of my use case: I have a table ‘projects’ in my database that shows various user-specific data in many columns on ONE row per project (efficient database design, considering the limitations of Airtable/Smartsuite that neither of them can combine multiple tables without automations ensuring endless linking), but this requires multiple blocks with filters in Softr (as Softr is unable to connect list blocks to more than one table (point 1 above) resulting in inefficient data loading/filtering with Softr).

3. Add a grouping functionality to list-blocks. A request was already made here: List / Table view - Grouping records - Ask the community - Softr Community - This can seriously reduce loading times (and make the website look SO MUCH better).

[thijs]

Dear community,

Since two weeks ago - I managed major progress in the responsiveness of my website. Earlier speed test showed that I could register my most complex configuration (of +100 choices while plowing through 40 groups with a total of 300 options) in approximately 45 minutes. That was not impressive, I first thought this was inevitable with no-code. But last week I got this down to 30 minutes (see my explanation how in earlier contributions above).
And as of this morning I managed to do this in 20 minutes !! Now my website feels quite snappy/responsive, instead of slow/laggy.

Softr and rest of community: it honestly surprises me that I seem to be the only one interested in performance optimization. At least I would expect Softr to be a huge supporter of this topic - and eager to make some kind of BEST PRACTICE documentation around this subject.
Note that I’m in the lucky position to be adviced on this matter (outside of this community) - which led me to these results, but that may not be the case for many other users.

Anyways, I found yet another major improvement in registering options from my menucard (that brought my total time from 30 minutes down to 20 minutes):

Old (and unreliable) way:

• ‘one-click-update’ to add an user name to a temporary field in Airtable - followed-up by Airtable automation to make the correct booking.
• automatic reload-triggers of tables required a delay of up to 4-5 seconds (and STILL be too early in about 3% of the time).
• perhaps a higher risk of Airtable being temporarily unavailable due to a pile-up/que of automations (small chance this happens at all, but I experienced this for the first time last week - that made the whole website unusable - and is a guaranteed bad user experience).

Much better way:

• a form to add ALL information at once (via hidden fields) directly in the relevant (junction)table in Airtable.
• better because: not only does this allow to record a personal choice (similar to one-click-update), but it also has the option to directly include any type of user-specified details (in my case: add to a wish list, add a requirement to specify the additional cost separately, any personal notes).
• better because: it is a simpler database design (with less data fields and less automations).
• faster and more reliable because: automatic reload-triggers of tables requires a delay of 1 second (and I did not notice any improper reload, perhaps this small delay is not even required).

My background/use case (again): I’m building something similar to a Dominos pizza configurator (but with a different subject) where up to +100 unique choices can be added to a personal configuration (form a list of 300 options in total).

[Tim_ClimatEU]

Thanks so much for looking into this @thijs , this is super relevant for me as well and I struggle with the speed of the Airtable API. I’ll look into this topic further soon as well, will get back to these posts then and share my learnings as well.

[Ben]

Great question for Softr builders. It reminds me of this post. Probably has significant implications on your app, no matter how you configure it or your base.

Speed of Softr webpages

@aar0n There are many Fortune100 companies using it. The question is what they are using it for and what’s the expectation. If you are looking for a website builder, many offer higher speeds as they focus on websites only and optimize for page speed etc… Softr is not a website builder, it allows you to build apps with non-trivial functionalities hence we are not able to dedicate ourselves full-on website optimization instead we make app building easier and give powerful features…

[nocodeking]

I wonder how the speed and performance vary by data source. We almost need to create a matrix and do a shootout with the same content but different back end to see how Airtable performs up against Smart Suite and Google Sheets in terms of API response times.

[thijs]

That would be very interesting to know. A noticeable difference in performance would be reason for me to switch.

Some months ago, I asked Smartsuite where they outperform Airtable. They did not mention anything about performance/speed. It did not seem like they could either outperform or underperform in this aspect, but it would indeed be interesting to test this. Although, that might take some effort (as you want a fair comparison which requires identical data/datastructure/Softr website/etc.).

[matthieu_chateau]

Hi all,

Until SQL (Supabase for example) or Xano types of backends are released, talking about performances with Airtable or Smartsuite (or worst… google sheets) will always be a dead end .

All of these softwares are not real backends, there are project management tools with API capabilities. Nothing more.

Softr structured the way it works according to the limits of these softwares. Without using caching system (except for user groups) in order to not retain sensitive data in their own servers (and to limit the server data costs I guess).

I heard SQL would come soon… And that the Softr complete system is currently being improved a lot…

Let’s see.

[matthieu_chateau]

Also, from the latest news, the API Rate limits for Gsheets => Batas penggunaan  |  Google Sheets  |  Google for Developers

For Airtable => 5 requests per second

[thijs]

Hi Matthieu,

That sounds promising. Has Softr announced this already?

I have never heard of Xano or Supabase, are those two ‘best in class’ for no-coders?j

[matthieu_chateau]

You can find them in the Softr roadmap.

Any of what I wrote has not being announced yet (as I wrote “I heard that”). And even if it would have been announced, it doesn’t change anything => it takes time to deploy all of this so don’t expect a specific date.

Xano and Supabase are best in class for no-coders yes. Supabase is more low-code.

Other no-code /low-code backends were launched sinceduring the past months (the number of these backend softwares will increase within the next 2 years).

For example fastgen.com, very promising or buildship.app

[jakepalm]

Thanks for all the research @thijs I find this very interesting as we are using both Softr for our website, but also as our internal SaaS to streamline our creative agency.

I am always looking to increase the speed both for the website, but also for the agents using Softr for their work. We recently moved some functionality to Airtable Interface designer as it was way faster.

Still love Softr and use it for a lot, and will look more into your points above to optimize internally.

[thijs]

So big surprise: Softr announced that their connection with Airtable is 3x faster now… How does that work exactly?

In my prior post (9 January) I measured a speed-run of my website (adding all available items) in 20 minutes consisting of (re)loading roughly 300 pages - of which 100 have a data-refresh delay of 1000 milliseconds - to enable Softr to obtain the adjusted values from Airtable. This old setup with a 1000 millisecond data-refresh delay resulted in a total time of 20 minutes and zero errors from a laggy connection with Airtable.

Today I adjusted the data-refresh delay to 333 milliseconds (because Airtable is now 3 times faster right?). This new setup with a 333 millisecond data-refresh delay resulted in a total time of 13 minutes and 2% errors from a laggy connection with Airtable (Airtable updated too late for refresh). That’s an improvement of 35% !!

The difference between a 1000 and 333 millisecond delay is like night and day - and was very much needed. Thanks for that - that seems to solve the performance for a big part.

But then again, not all performance issues are caused by Airtable - and perhaps there’s still more to gain from a smarter connection (of filters in list blocks and reload triggers) with Airtable, so the need for best practices on how to (further) improve the performance of Softr websites remains (especially for mobile).

[Jjenglert]

Thanks for following up on this @thijs and for reporting your findings. Very cool to see! We will never stop improving our performance, and as @matthieu_chateau said previously, we have some data sources coming that if could also provide a significant advancement in performance as well. Exciting times to say the least!

[thijs]

Hi @Jjenglert - yes sure, but aside any anticipated improvements I was asking for a response to my only request from you/Softr for a ‘best practice’ on how to (further) improve performance?!

If you read the responses from Softr in this thread - there’s nothing even hinting to acknowledgement that this might be helpful.

[Jjenglert]

I see, sorry I missed that originally @thijs

One thing that comes to mind is upgrading your airtable connection to a personal access token, allowing you faster speeds than Oauth. Have you tried that yet?