Firestore simple collectionGroup makes the function timeout

firestore query
firestore get document by id
get all documents in collection firestore
flutter firestore query
firestore subcollection
firestore getall
firestore get field value
firestore:delete all documents in collection

I have a cloud function that is scheduled to run every 2 mins. I am currently testing the function using an Http one, and trying to run a simple collectionGroup query as follows

Update 1 I updated my code to handle offset and limit but now it's working fine for the first record by then hangs while fetching the second record.

please check the snippet

/* eslint-disable promise/no-nesting */

/**
 * @description
 * This module will send out emails to different users who have alerts not sent out yet
 * @algorithm
 * 1 - Get all alerts where sent = false 
 * 2 - Send out those alerts via the email module. 
 */
const functions = require("firebase-functions");
const { db } = require("../helpers/firestore");
// console.log('admin: ', admin);
let _allUserAlerts = [];
let counter = 0;

exports.sendEmailsToUsers = functions.pubsub
  .schedule("every 2 minutes")
  .onRun(context => {
    return sendEmailsToUsers(context);
  });

exports.testSendEmailsToUsers = functions.https.onRequest((req, res) => {
  return sendEmailsToUsers("hello");
});

async function sendEmailsToUsers(context) {
  try {
    console.log("function execution")
    let alerts = await db
      .collectionGroup("alerts")
      .where("sent", "==", false)
      .limit(1)
      .offset(counter)
      .get();

    alerts.forEach(_alert => {
      if (_alert.exists) {
        console.log("alert exists")
        _allUserAlerts.push(_alert.data());
        counter++;
        return sendEmailsToUsers()
      } else {
        console.log("alert ends")
        sendAlertsToEmail(_allUserAlerts)
      }
    })
  } catch (err) {
    console.log(err)
  }
}

async function sendAlertsToEmail(alerts) {
  console.log('alerts: ', alerts);
}

Old code

/* eslint-disable promise/no-nesting */

/**
 * @description
 * This module will send out emails to different users who have alerts not sent out yet
 * @algorithm
 * 1 - Get all alerts where sent = false 
 * 2 - Send out those alerts via the email module. 
 */
const functions = require("firebase-functions");
const { db } = require("../helpers/firestore");
const { Email } = require("../EmailModule");

const { groupBy } = require("../helpers/groupBy");

exports.sendEmailsToUsers = functions.pubsub
  .schedule("every 2 minutes")
  .onRun(context => {
    return sendEmailsToUsers(context);
  });

async function sendEmailsToUsers(context) {
  try {
    console.log("function execution")
    let alerts = await db
      .collectionGroup("alerts")
      .where("sent", "==", false)
      .get();
    console.log('alerts: ', alerts);
  } catch (err) {
    console.log(err)
  }
}

This is literally the entire code in the function, but the function times out and I don't get to the log() of alerts. I am not getting what could be the problem here ? Any thing I might be missing or not getting here ? Any help would be appreciated.

Thanks

Maybe the process simply takes more than 60s. You can increase the timeout of the cloud function. You can raise it up to 9 minutes and it should work.

Otherwise the problem does not seems to be from the function itself. The issue may be somewhere here : const { db } = require("../helpers/firestore"); . "db" is not coming, and that's way your code stops at let alerts = await db .

Firestore: collectionGroup with where clause causes execution , Firestore: collectionGroup with where clause causes execution timeout error # 2542. Open. ulver2812 opened this issue on Jan 8 � 0 comments. Open� Starting functions v2.0.0 you can also set the timeout in your function declaration as described in the doc under the "Set timeout and memory allocation" section:. const runtimeOpts = { timeoutSeconds: 300, memory: '1GB' } exports.myStorageFunction = functions .runWith(runtimeOpts) .storage .object() .onFinalize((object) = > { // do some complicated things that take a lot of memory and time });

It sounds like you might be requesting too much data to transmit and store in memory without problems. I suggest paginating your data with limit() instead so that you can batch process results without exceeding the limits of Cloud Functions server instances (which are quite limited). Start small and work your way up.

Delete data with a Callable Cloud Function, Get data once � Listen for realtime updates � Perform simple and compound queries Deleting a document does not delete the documents in its subcollections. logic for your Cloud Function, you can take advantage of the firestore:delete command Timeouts - the function above is configured to run for a maximum of 540� We can use the simple and compound query described earlier to query a single city's landmarks subcollection, but you might also want to retrieve results from every city's landmarks subcollection at once. The landmarks collection group consists of all collections with the ID landmarks, and you can

The solution to this problem was to upgrade the current node version. By default Cloud functions work on Node 8 which has stopped support from the official node.js team. So when I upgrade to Node 10 it started working again perfectly !

Perform simple and compound queries in Cloud Firestore, For the web and mobile SDKs, you must also create rules that allow your collection group queries. Query limitations. Cloud Firestore either does not support or� This typically means you're using a version of the Admin SDK that doesn't support Collection Group Queries yet. Support for these was introduced in version 7.4.0, based on Cloud Firestore client 1.3.0.

Best practices | Firestore, A simple way to handle this migration is to read from the old collection, and if the document does not exist, then read from the new collection. This section describes how to secure your collection group queries using security rules. Secure and query documents based on collection groups. In your security rules, you must explicitly allow collection group queries by writing a rule for the collection group: Make sure rules_version = '2'; is the first line of your

Querying and filtering data | Firestore, Example data; Simple queries. Execute a query; Query operators After creating a query object, use the get() function to retrieve the results: Web Swift More. Events are delivered at least once, but a single event may result in multiple function invocations. Avoid depending on exactly-once mechanics, and write idempotent functions. Cloud Firestore triggers for Cloud Functions is available only for Cloud Firestore in Native mode. It is not available for Cloud Firestore in Datastore mode.

“Why is my Cloud Firestore query slow?”, How to make this faster. The best way to fix this issue is to make sure you're not transferring down more data than you need. One simple option is� See Reporting Errors for more discussion of how to report errors in Cloud Functions. Timeout. Function execution time is limited by the timeout duration, which you can specify at function deployment time. By default, a function times out after 1 minute, but you can extend this period up to 9 minutes.

Comments
  • That is not literally the entire code, at least not in the sense that would be helpful for use on Stack Overflow. Please edit the question to include the function definition as well - you should include the entire contents of the file so we can see what kind of function this is, and explain how you are invoking it. An MCVE will be helpful. Please read: stackoverflow.com/help/minimal-reproducible-example
  • Okay I've added the function definition as well. See if there's an anomaly ?
  • What exactly is in the log, if not what you expect?
  • It doesn't matter. The problem is the function just stops on let alerts = await db part and times out after 60 seconds. The console part is just to see if the code is running or not.
  • Could you share the log anyway, even if you think it doesn't matter?
  • I tried extending function time to 9 minutes but still it timed out. The strange part is that it runs fine for some collection and for some it just times out Also I am getting this in the logs: NOTICE: Longjohn is known to cause CPU usage due to its extensive data collection during runtime. It generally should not be used in production applications.
  • Maybe you are hitting the quotas for CF? Take a look on this list : cloud.google.com/functions/quotas
  • I just did that, it works fine for the first document but then hangs when fetching the next one. I've updated my question for the updated code.