| 2023-02-15

Google Cloud Platform Uploader

    Introduction

    This app is a simple web application that provides file upload functionality to a Google Cloud Storage bucket. It uses the Express framework to define API endpoints for uploading, deleting, and checking files in the bucket. Multer is used for processing file uploads, and the @google-cloud/storage package is used for communicating with the Google Cloud Storage API. The app uses the cors package to enable CORS (Cross-Origin Resource Sharing) for all routes.

    Prerequisites

    Before we start building the app, make sure that you have the following installed:

    • Node.js and NPM (Node Package Manager) installed on your system
    • A Google Cloud Storage bucket set up

    You will also need to have a basic understanding of Node.js and Express.

    Setting up the project

    To get started, create a new directory for your project and navigate to it in the terminal. Then run the following command to create a new Node.js project:

    npm init -y
    
    bash

    This will create a new package.json file in the project directory.

    Next, install the following packages:

    • express
    • @google-cloud/storage
    • multer
    • dotenv
    • cors

    You can install these packages using NPM with the following command:

    npm install express @google-cloud/storage multer dotenv cors
    
    bash

    We will be using environment variables to store sensitive information such as our Google Cloud Storage credentials. Create a new file in your project directory called .env and add the following variables:

    GOOGLE_CLOUD_PROJECT_ID=your-project-id
    GOOGLE_CLOUD_PRIVATE_KEY_ID=your-private-key-id
    GOOGLE_CLOUD_PRIVATE_KEY=your-private-key
    GOOGLE_CLIENT_EMAIL=your-client-email
    GOOGLE_CLIENT_ID=your-client-id
    GOOGLE_CLIENT_X509_CERT_URL=your-client-cert-url
    GOOGLE_CLOUD_STORAGE_BUCKET=your-bucket-name
    PORT=8080
    
    bash

    Replace the placeholders with your own values. You can find your project ID, private key ID, private key, client email, and client ID in the JSON file that you downloaded when you created your Google Cloud Storage service account. The client X.509 certificate URL can be found in the service account details. The bucket name is the name of the bucket that you created in Google Cloud Storage. Finally, the port number is the port that the app will listen on.

    Creating the app

    Now that you have the necessary packages installed, create a new file named app.js in your project directory, and paste the following code into it:

    // Import required modules
    const express = require("express");
    const multer = require("multer");
    const { Storage } = require("@google-cloud/storage");
    require("dotenv").config();
    const cors = require("cors");
    
    //CORS is a mechanism that allows restricted resources on a web page to be requested from another domain outside the domain from which the first resource was served. In other words, it allows you to make requests to a server from a different domain. This is useful for web applications that are hosted on different domains, but need to communicate with each other. For example, if you have a web app hosted on a domain like example.com, and you want to make requests to a server hosted on a different domain like api.example.com, you will need to enable CORS for the API server.
    
    // Create a new Express app
    const app = express();
    
    // Enable CORS (Cross-Origin Resource Sharing) for all routes
    app.use(cors());
    
    // Create a new Storage instance using the credentials from the environment variables
    const storage = new Storage({
      projectId: process.env.GOOGLE_CLOUD_PROJECT_ID,
      credentials: {
        type: "service_account",
        project_id: process.env.GOOGLE_CLOUD_PROJECT_ID,
        private_key_id: process.env.GOOGLE_CLOUD_PRIVATE_KEY_ID,
        private_key: process.env.GOOGLE_CLOUD_PRIVATE_KEY,
        client_email: process.env.GOOGLE_CLIENT_EMAIL,
        client_id: process.env.GOOGLE_CLIENT_ID,
        auth_uri: "https://accounts.google.com/o/oauth2/auth",
        token_uri: "https://oauth2.googleapis.com/token",
        auth_provider_x509_cert_url: "https://www.googleapis.com/oauth2/v1/certs",
        client_x509_cert_url: process.env.GOOGLE_CLIENT_X509_CERT_URL,
      },
    });
    
    // Define a route to handle GET requests to the root URL
    app.get("/", (req, res) => {
      // Send a response with a dynamic message including an environment variable
      res.send(`Hello, ${process.env.SOME_VAR}!`);
    });
    
    // Get the bucket object using the environment variable
    const bucket = storage.bucket(process.env.GOOGLE_CLOUD_STORAGE_BUCKET);
    
    // Create a new multer instance to handle file uploads
    const upload = multer({
      // Use in-memory storage for uploaded files
      storage: multer.memoryStorage(),
      // Limit the file size to 500MB
      limits: {
        fileSize: 500 * 1024 * 1024,
      },
    });
    
    // Define a route to check if a file exists in the bucket
    app.get("/check", async (req, res, next) => {
      try {
        // Get the filename from the query string
        const filename = req.query.filename;
    
        // If the filename is not provided, return an error response
        if (!filename) {
          return res.status(400).send("Filename parameter is missing.");
        }
    
        // Get a reference to the file in the bucket
        const file = bucket.file(filename);
        // Check if the file exists in the bucket
        const [exists] = await file.exists();
    
        // Send a response with the result of the existence check
        res.status(200).send({ exists });
      } catch (err) {
        // Pass any errors to the next middleware
        next(err);
      }
    });
    
    // Define a route to handle file uploads
    app.post("/upload", upload.single("file"), async (req, res, next) => {
      try {
        // If no file was uploaded, return an error response
        if (!req.file) {
          return res.status(400).send("No file uploaded.");
        }
    
        // Create a reference to the file in the bucket with the same name as the uploaded file
        const blob = bucket.file(req.file.originalname);
    
        // Create a write stream for the file in the bucket
        const blobStream = blob.createWriteStream({
          resumable: false,
        });
    
        // Handle any errors that occur while writing the file to the bucket
        blobStream.on("error", (err) => {
          next(err);
        });
    
        // Handle the completion of the file upload
        blobStream.on("finish", async () => {
          const publicUrl = `https://storage.googleapis.com/${bucket.name}/${blob.name}`;
          res.status(200).send(`File uploaded to: ${publicUrl}`);
        });
    
        // Get the public URL of the uploaded
        blobStream.end(req.file.buffer);
      } catch (err) {
        next(err);
      }
    });
    
    // Define a route to handle file deletions
    app.delete("/delete", async (req, res, next) => {
      try {
        const filename = req.query.filename;
    
        // If the filename is not provided, return an error response
        if (!filename) {
          return res.status(400).send("Filename parameter is missing.");
        }
    
        // Get a reference to the file in the bucket
        const file = bucket.file(filename);
        // Delete the file from the bucket
        await file.delete();
    
        // Send a response with a success message
        res.status(200).send({ message: "File deleted successfully." });
      } catch (err) {
        next(err);
      }
    });
    
    // Define a route to handle file downloads
    const PORT = process.env.PORT || 8080;
    
    // Start the server
    app.listen(PORT, () => {
      console.log(`Server listening on port ${PORT}.`);
    });
    
    javascript

    To launch the app, run the following command:

    node app.js
    
    bash

    Thanks for reading. If you enjoyed this post, I invite you to explore more of my site. I write about web development, programming, and other fun stuff.