Image uploads filling Heroku log output

Hello,

We are running parse server on Heroku, and having problem with some log output.
We use a plugin (paperTrail) to save the logs of Heroku to check errors etc, but every time some images are uploaded, it fills up our daily quota of paperTrail pretty quickly (because it outputs a big binary of image file).

We looked into filtering function of paperTrail, but when an image is uploaded, it starts with generic keyword ā€œapp[web.1]ā€, so we cannot just only filter the logs of image binary output (all the other logs starting with ā€œapp[web.1]ā€ is not saved anymore, as expected).

Also, it does not help to filter out ā€œinputā€ because image binary output is too long, so it gets truncated, and right after, the rest of the binary data log is printed out with ā€œapp[web.1]ā€ multiple times.

Do you know if there is a way not to log output of images or change the starting words ā€œapp[web.1]ā€ only for image output?
Thanks!

Are you running with verbose option set true? I am not sure Parse Server automatically logs out image contents (actually even with verbose true). How are you uploading these images?

Hello, thanks for your reply!
We didnā€™t set ā€œverboseā€, so we just tested with ā€œverbose: falseā€ in index.js, but it still shows big logs on heroku when any image is uploaded.
We upload image by cloud code (to AWS) by something like:

const imageTest = request.params.imageTest;
var image1= new Parse.File(ā€œimage.jpgā€, imageTest);
post.set(ā€œimageā€, image1);

Do you have any idea why??

Yes. You are sending the content of the image as a cloud code function parameter. And thatā€™s why it is being logged. Why donā€™t you save the file in the client and then just send to the cloud code function the file object? It should fix your problem.

Oh, I see! I tried to find how to do what you suggested: ā€œsave the file in the client and then just send to the cloud code function the file objectā€, but could not figure out how to make this work.

From client side (iOS in the following case), we save parse file and set the cloud code parameter as

if let image:UIImage = self.photoView?.image,
let data:Data = image.jpegData(compressionQuality: 0.95),
let imageFile:PFFileObject = PFFileObject(data: data){

      params["image"] = imageFile

}


Then in the Cloud function,

we receive this Parse file by

const image = request.params.image;

Then, set

newPost.set(ā€œimageTestā€, image);

But then we get error: ā€œCannot read properties of undefined (reading ā€˜formatā€™)ā€

during saving the file: await newPost.save(null, { useMasterKey: true });

Would you have some reference how to do this? I guess the way I am passing parse file to cloud code is not correct (since I am still passing the parse file as a cloud code function parameter).

(We used to create Parse.File on client side, and upload it directly to parse server, but now, we turned off all the Class Level Permissions for this collection for security, so we have to go through cloud function to save images.)

You have to save the file (your imageFile var) in the client, before calling the cloud function.

Thanks for your reply!
I am sorry we still donā€™t understand how to do what you are suggesting.
Do you mean we should save the parse file in the local database of the client device?
(we are saving these image files (PFObjects) to AWS through parse server)
Could you give me some reference or a few example lines for this?

You need to call imageFile.saveInBackground() before calling the cloud code function passing the imageFile as parameter. Would you mind to share the code that you are using including the cloud code function call?

Hello,

Thanks for your reply!!

=====================================================
What we have been doing is the following:
(This works fine, except we have the big binary logs on Heroku.)

From client device (iOS), we pass parameter as:

if  let image:UIImage       = self.photoView1?.image,
    let data:Data = image.jpegData(compressionQuality: compressionQL)
   {
                    params["image1"] = data


//Then we call cloud function "newPostFun"

PFCloud.callFunction(inBackground: "newPostFun", withParameters: params, block: { (result: Any?, error: Error?) -> Void in

In the cloud code function:

Parse.Cloud.define("newPostFun", async (request) => {

   const image1 = request.params.image1;
   var newPost = new Parse.Object('Post');
   
  
    var image1PF= new Parse.File("image.jpg", image1);
     newPost.set("image1", image1PF);
      

   //... set other parameters as well  ...

   await newPost.save(null, { useMasterKey: true });

});

This works fine (except we get big binary heorku logs that fills up paperTrail).

=====================================================
What we have tried now, after adding a line " imageFile.saveInBackground() " is:

From client device (iOS), we pass parameter as:

 if  let image:UIImage       = self.photoView1?.image,
     let data:Data = image.jpegData(compressionQuality: 0.95),
      let imageFile:PFFileObject    = PFFileObject(data: data)
       {
                    imageFile.saveInBackground() 
                    params["image1"] = imageFile

//Then we call cloud function "newPostFun" (the same as before)

PFCloud.callFunction(inBackground: "newPostFun", withParameters: params, block: { (result: Any?, error: Error?) -> Void in

In the cloud code function:

Parse.Cloud.define("newPostFun", async (request) => {

   const image1PF = request.params.image1;

   var newPost = new Parse.Object('Post');

   newPost.set(ā€œimage1ā€, image1PF);

   //... set other parameters as well  ...

   await newPost.save(null, { useMasterKey: true });

});

This gives error on Heroku at the last line while trying to save the Post object, "await newPost.save(null, { useMasterKey: true });"

2022-07-26T07:23:07.72307+00:00 app[web.1]: error: Cannot read properties of undefined (reading ā€˜formatā€™) {ā€œcodeā€:141,ā€œstackā€:ā€œTypeError: Cannot read properties of undefined (reading ā€˜formatā€™)\n at ParseFile.save (/app/node_modules/parse-server/node_modules/parse/lib/node/ParseFile.js:375:24)\n at /app/node_modules/parse-server/node_modules/parse/lib/node/ParseObject.js:3088:30\n at Array.forEach ()\n at Object.save (/app/node_modules/parse-server/node_modules/parse/lib/node/ParseObject.js:3086:15)\n at ParseObject.save (/app/node_modules/parse-server/node_modules/parse/lib/node/ParseObject.js:1817:23)\n at /app/cloud/main.js:1776:17\n at processTicksAndRejections (node:internal/process/task_queues:96:5)ā€}

On the client side, the error is
ā€œCannot read properties of undefined (reading ā€˜formatā€™)ā€

=====================================================

Do you see some obvious mistake we are making??

Yes. You should await the saveInBackground to complete before calling the cloud code function. Something like this:

imageFile.saveInBackground({ (success: Bool, error: Error?) in
    // Handle success or failure here ...
   params["image1"] = imageFile
   PFCloud.callFunction(inBackground: "newPostFun", withParameters: params, block: { (result: Any?, error: Error?) -> Void in
})

Hello,
Sorry, for the slow reply.
Yes, that seems to work.
Thanks so much!!

Actually, I have another question related to thisā€¦
When we save the image file on device with

imageFile.saveInBackground({

it takes too much time (more than a couple of seconds per image) to complete, which seems strange, considering it is only saving it on the same device. Could you think of what the cause of this delay?

We are wondering if it tries to access to the remote parse server (on Heroku in our case) first, and it waits until the response comes back (it should fail because our parse serverā€™s CLP setting does not allow to save files this way), and afterwards, it tries to save it locally on the device. Could this be the reason why it takes such a long time to complete ā€œsaveInBackgroundā€ on device? If so, could you think of any ways avoid this time lag?

Thanks a lot!!

saveInBackground function sends the file to the server.

Ok, we finally started to understand how this PFFileObject things work. Thanks a lot!!