Benchmark scores are lower than expected?

#1

Hello. İ have a server with 4 cpu and 8gb ram. Server uses AMD Epyc gen 2 cpu. My cpu gets 800 point for single core and 3000 points for multicore from GeekBench 5 test. So my cpu is not weak. İt’s actually pretty good according to GeekBench

İ decided to make a benchmark. İ used wrk library.

İ made request to my cloud code using functions endpoint. You can see my cloud code here

Results are lower than i expected. İ get 60-70 requests per second i expected at least 200.

İ have an Nginx reverse proxy server on front of my s3 storage. This server only has 2gb ram and 2cpu. İ cache images in this server. With same test, i get 5000 request per second for larger files and i once even get 14000 request for small files. İ know static files will be much faster. But gap between them are way higher. Ever my parse server is more powerful.

Yes parse server is dynamic server and has to recreate response Everytime. But still it looks lower.

İ even tried with 16cpu and 32gm ram machine. But highest value i get was 550 req/sec.

I’m using pm2 module. Pm2 runs parse server in cluster mode and load balancer between them.

İ also copied parse server to different folder and launched different parse server in different ports in same machine. For example
127.0.0.1:1338
127.0.0.1:1339
127.0.0.1:1340
127.0.0.1:1341

And load balanced them with Nginx to see if problem is related to pm2. But it’s not. İ get same results.

İs this results are normal? İf not, what am I doing wrong here?

Edit: Additional information;
Pm2 runs 4 instances of parse server, because i have 4 cpu.

My parse server version is 4.2.0
My MongoDb version is 4.2.8

What someone should do to increase performance of Parse Server?
Should we direct questions to Server Fault?
#2

The bottleneck can be in the database. Where is your database hosted? Do you have indexes for the queries you are performing?

#3

İ don’t have any indexes. But my database has only 13 objects for each class. So i don’t think database is bottleneck.

My database is in remote server. (3cpu and 4gb ram).

Edit: When I first started parse-server, I get 200 req/sec. But this value decreases. Second test gives 150 req/sec. 3rd test is 100req/sec. 4th test gives 80 etc…

First I thought its ram issue. But parse server only uses 2.5gb of total ram. and 6.5gb ram is still empty. I have 4 parse server running so each parse server only uses 500 - 600mb of ram.

Also I created indexes. And result still same.

I also did two simoultaneous test with different session tokens to see if parse server blocking too many requests from same user. But results are still same

#4

What is the latency between Parse Server and MongoDB?

#5

I dont know. How can I learn that?

But I dont think this is database issue. Because when I do same benchmark with 16 core cpu and 32gb ram machine(They both connected to same database) I get 500 req/sec(Which is too low for the hardware).

And I also discovered this happens only if I benchmark cloud code. When I do benchmark over classes endpoint for example parse/classes/Post, benchmark results are steady, It always return same result with a little difference. for example first test 650, second 630, third 680 etc.

Also Can you tell me what is the expected req/sec result for my case, Whats the ideal result for 4cpu and 8gb ram? I see that Back4App has dedicated plans with same specs(4 cpu and 8 ram). Whats the normal req/sec you get in those plans?

#6

It really depends on the type of request that the app is performing and a cloud code function for sure will perform worse than a regular call. Here it goes some ideas that you can consider to test:

  • Parse server experimental configuration option directAccess (it should speed up your cloud code execution)
  • Other Parse Server configuration options such as cacheTTL, schemaCacheTTL, enableSingleSchemaCache, verbose (should not be verbose in production), logLevel, silent
  • MongoDB connection parameters, such as connection pool size, max timeout, etc
  • Indexes in the MongoDB
  • Latency between client and application server
  • Latency between MongoDB and application server
  • Bandwidth between client/application server and between application server/MongoDB
  • Bandwidth of application server’s and MongoDB HDs.
1 Like
#7

Thank you. I enabled enableSingleSchemaCache etc.

But I couldnt find anything about mongodb pool size. How can I change that. And Whats the default pool size?

Edit: Hey @davimacedo I think I found the answer. Slowness only happens when parse server returns succesfull response. But if my cloud code throws an error like this:

Parse.Cloud.define("exampleCloudCode", async (request) => {

  throw "error";

});

My server can handle thousands of cloud code operation. So cloud code doesnt slow proccess itself.

I measured latency between my server and my database with ping command, I got 21 ms latency. My server located in germany and database located in britain.

I do 3 get operation in my cloud code. And 21*3 = 63ms latency per cloud code.

if we divide 1000ms to 63 we get 15 operations per second. And I have 4 instances of parse server in my machine, 15*4 =60 request per second.

I think this is my problem. I will move my database to germany and connect to my server with private networks. With this way, I get 0.4 ms latency which is 50 times faster than london location.

I’m not sure if this is my problem. But I will give it a try.

Edit2: Now after some thinking, I have doubts. I mean if my theory is true, then that means we can only execute one cloud code at same time. Which sounds wrong. I’m really confused.

Edit3: I tried to move my database to near of my server. There is only 0,4ms latency between them. But I still get same number of req/sec. Its not about latency. Only way to increase rec/sec ratio is upgrading to better hardware. Only if I increase cpu cores, req/sec will increase. Even 16 core 32gb machine only gives 450 req/sec. I dont know if this is about node js being single thread or parse server designed this way. I couldnt increase performance of my server

Edit4: i need to mention. İt takes at least 8 second to return response. For example user get the response after 8 seconds after calling cloud code. And this is minimum time. Maximum time is 25 second. So some of the user will get response after waiting 25 seconds. Previous cloud codes definitely blocking later processes.

#8

You probably have something wrong with your cloud code function. Maybe you should try to optimize it.