I have some cloud functions which mostly run some queries and return the results of the queries. I need these queries in cloud code because I have multiple clients written in different languages and I don’t want to write the same query multiple times.
Let’s assume that one of the clients is a typescript-based web app using the JS SDK. My problem is that I need on the client side to fetch the results in plain JS objects/JSON format and not as arrays of Parse.Objects. I can’t seem to find a built-in way to do this, it looks like the Parse.Cloud.run function runs an encode function which converts the result to Parse types and I can’t find any parameter or option I can use to avoid this call.
My current workaround is, instead of doing something like:
return await query.find();
in the cloud function, I do:
return JSON.stringify(await query.find());
This basically returns a JSON string which I can easily JSON.parse on the client side and get a plain JS object that I can feed to something like https://zod.dev to validate, type infer etc.
Is there a better way to do this? Does anyone know if this approach has any major downsides, especially when returning large amounts of data?
You can use this in server side.
const results= await query.find();
const jsonResults = results.map(obj => obj.toJSON());
return jsonResults
The toJSON() method requires less processing than converting all data to string with JSON.stringify. It eliminates the client-side JSON.parse step, increasing speed for large data sets.
1 Like
@rgunindi oh nice, really good idea. Thanks!
@rgunindi Seems like the only problem with this solution is that nested object(eg. included pointers) get returned as Parse.Objects, so they need to be handled separately.
Just tested with return await query.find({ json: true }); and it has the same problem with the nested objects. The objects in the results array are plain JS objects, but any nested object becomes a Parse.Object.
I think this might actually be a bug in the SDK.
Try with query.toJSON(), and don’t need the json: true on the “find”, also don’t need the obj.toJSON() at the “map”.
Source: Query - Documentation
I think query.toJSON serialises the query itself and not the results of the query. The result is basically an array of Parse.Objects if json:true is not passed or an array of plain objects if it is.
The problem is that the map method only calls the toJSON function on the objects in the array and it does not recursively call it on the nested objects. I’d probably need to manually do that, not sure if there is a straight, one-liner JS way of doing it.
With the query json option it seems that a __type: 'Object' field is added to the nested objects and that causes the encoder in Parse.Cloud.run function to convert those nested objects into Parse.Objects. So this method would work well if you’d have the option to fetch the results directly from a query, but it doesn’t work if you “wrap” that query in a cloud function because of the encoder call from Parse.Cloud.run.
A solution like this, where we iterate recursively and just remove the __type field instead of recursively calling toJSON(like we could do with the map solution):
// In Cloud code
function removeType(obj) {
for (prop in obj) {
if (prop === '__type')
delete obj[prop];
else if (typeof obj[prop] === 'object')
removeType(obj[prop]);
}
}
...
const results = await cardsQuery.find({ json: true });
removeType(results);
return results;
Works quite well(and can probably be further optimised), but it’s not pretty and it would be ideal to be able to reach the same result just by using the json option in the Query.find function.
I’m still not sure if the added __type field is a bug or not, since it seems to only appear in the nested objects(eg. included pointers) and does not appear in the top level objects in the results array…
@BobyIlea I like this feature and had this idea a while ago. Can you open a request on Parse Server?
The encoding happens here. I don’t know if we want an encode: false option on Parse.Cloud.run or a validator option on Parse.Cloud.define. I perfer the latter as it doesn’t require a client side update. We can open it up for discussion.
Yeah I will. I’m also not really sure if it should be an update to the SDK or to the validator mechanic, both options might prove useful in different scenarios.
@dplewis what do you think about the way that json option currently works in Query.find for nested objects? Do you think it’s intentional or a bug?
json options prevents encoding client side so you can access fields directly.
object.get(‘myField’) → object.myField or parent.get(‘child’).get(‘myField’) → parent.child.myField. This is intentional but I can see how _type could be a problem in your case as there is still encoding involved which means it’s a bug. I think__type comes directly from the server but more research is needed.
@dplewis my guess is that it’s a bug simply because __type is missing from the top level objects. It is being added only to the nested ones. So I believe that the behaviour should be consistent at all levels. But this is an issue in the Query class and not really related to cloud code. Though, if fixed, it can easily be used instead of the other two proposed solutions(validator and option in the run function).
It is missing from the top level, even if added run will also decode the top level object into an Parse.Object too. Maybe I’m missing something and do more research. Feel free to open a PR/issue on the JS SDK.
@dplewis for a more concise example, this query:
query.include(["a", "a.b"]);
query.find({ json: true });
produces this result:
[{
objectId: "...",
className: "M",
a: {
objectId: "...",
className: "A",
__type: "Object",
b: {
objectId: "...",
className: "B",
__type: "Object",
}
}
}, ... ]
If we fetch this result from a cloud function:
const result = await Parse.Cloud.run(...);
const m = result[0];
We’ll see that Cloud.run only converts nested objects m.a and m.a.b to Parse.Objects, while object m itself, and all other objects in the results array remain plain objects.
This is because object m is missing __type field, while the nested objects do have it. And the decoder seems to check for __type and className before decoding:
This is why my “hacky” solution to recursively iterate and remove all __type fields actually works. But it would be much better if m, m.a, m.a.b etc. all had the same format…
I see json: true should return the same result as object._toFullJSON(). I can add a fix for this but you will still need your “hacky” solution right?
I believe the right solution would be to call Object.toJSON on all objects, including the nested ones. Now if you mention it, it does seem that on nested objects Objects._toFullJSON is called instead. But I can’t seem to find the place where this happens.
That would be too much overhead and impact performance just disabling decoding in Parse.Cloud.run should be enough. I’ll do a PR for it.
1 Like
Problem with fixing the run function is that it will need to be adjusted in all SDKs to have consistent behaviour. A fix in the code that runs on the server would be better. I’ll think a bit more about where what would be the best way to tackle this.
Also, I don’t think that fixing the toJSON call will incur overhead. I think that is already happening, just that it’s happening with toFullSJON that’s why it’s appending the __type field. But I can’t say for sure since I don’t seem to understand exactly the mechanics of include.