Hot questions for Using Vapor in database

Question:

I want to write some integration tests for Vapor 3 server and I need to have clean Postgre database each time I run my tests. How can I achieve this? It seems migrations isn't the right way to go as they've been running once if database doesn't exist yet.


Answer:

Have a look at https://github.com/raywenderlich/vapor-til/tree/master/Tests

This requires a DB to be running before you run the tests, but it reverts all migrations at the start of each test run, which gives you a clean DB each time. (Specifically here)

There's also a docker-compose.yml in the root directory for spinning up a completely isolated test environment on Linux

Question:

I want to create a command in which you can create a user (like database seeds).

However, I cannot access a database in a command, my code is like the following:

import Command
import Crypto

struct CreateUserCommand: Command {
    var arguments: [CommandArgument] {
        return [.argument(name: "email")]
    }

    var options: [CommandOption] {
        return [
            .value(name: "password", short: "p", default: "", help: ["Password of a user"]),
        ]
    }

    var help: [String] {
        return ["Create a user with provided identities."]
    }

    func run(using context: CommandContext) throws -> Future<Void> {
        let email = try context.argument("email")
        let password = try context.requireOption("password")
        let passwordHash = try BCrypt.hash(password)
        let user = User(email: email, password: password)
        return user.save(on: context.container).map(to: Future<Void>) { user in
            return .done(on: context.container)
        }
    }
}

Like the above, I want to save users by executing a query on context.container, but I got argument type 'Container' does not conform to expected type 'DatabaseConnectable' Error.

How to access to the database in a command?


Answer:

It seems like this might be the way to go:

func run(using context: CommandContext) throws -> EventLoopFuture<Void> {
    let email = try context.argument("email")
    let password = try context.requireOption("password")
    let passwordHash = try BCrypt.hash(password)
    let user = User(email: email, password: password)

    return context.container.withNewConnection(to: .psql) { db in
        return user.save(on: db).transform(to: ())
    }
}

Question:

I want to be able to bulk add records to a nosql database in Vapor 3.

This is my Struct.

struct Country: Content {

   let countryName: String
   let timezone: String
   let defaultPickupLocation: String

}

So I'm trying to pass an array of JSON objects but I'm not sure how to structure the route nor how to access the array to decode each one.

I have tried this route:

    let countryGroup = router.grouped("api/country")

    countryGroup.post([Country.self], at:"bulk", use: bulkAddCountries)

with this function:

 func bulkAddCountries(req: Request, countries:[Country]) throws ->  Future<String> {
    for country in countries{
    return try req.content.decode(Country.self).map(to: String.self) { countries in



        //creates a JSON encoder to encode the JSON data
        let encoder = JSONEncoder()
        let countryData:Data
        do{
            countryData = try encoder.encode(country) // encode the data
        } catch {
            return "Error. Data in the wrong format."
        }
        // code to save data
    }
    }
}

So how do I structure both the Route and the function to get access to each country?


Answer:

I'm not sure which NoSQL database you plan on using, but the current beta versions of MongoKitten 5 and Meow 2.0 make this pretty easy.

Please note how we didn't write documentation for these two libraries yet as we pushed to a stable API first. The following code is roughly what you need with MongoKitten 5:

// Register MongoKitten to Vapor's Services
services.register(Future<MongoKitten.Database>.self) { container in
    return try MongoKitten.Database.connect(settings: ConnectionSettings("mongodb://localhost/my-db"), on: container.eventLoop)
}

// Globally, add this so that the above code can register MongoKitten to Vapor's Services
extension Future: Service where T == MongoKitten.Database {}

// An adaptation of your function
func bulkAddCountries(req: Request, countries:[Country]) throws ->  Future<Response> {
    // Get a handle to MongoDB
    let database = req.make(Future<MongoKitten.Database>.self)

    // Make a `Document` for each Country
    let documents = try countries.map { country in
        return try BSONEncoder().encode(country)
    }

    // Insert the countries to the "countries" MongoDB collection
    return database["countries"].insert(documents: documents).map { success in
        return // Return a successful response
    }
}

Question:

I am trying to fix an error I have been getting recently when I run my Vapor project.

It builds fine, but when it runs, it crashes. Here is my log:

fatal error: Error raised at top level: Fluent.EntityError.noDatabase: file /Library/Caches/com.apple.xbs/Sources/swiftlang/swiftlang-800.0.58.6/src/swift/stdlib/public/core/ErrorType.swift, line 184
Current stack trace:
0    libswiftCore.dylib                 0x0000000100fe7cc0 swift_reportError + 132
1    libswiftCore.dylib                 0x0000000101004f50 _swift_stdlib_reportFatalErrorInFile + 112
2    libswiftCore.dylib                 0x0000000100fb3370 partial apply for (_assertionFailed(StaticString, String, StaticString, UInt, flags : UInt32) -> Never).(closure #1).(closure #1).(closure #1) + 99
3    libswiftCore.dylib                 0x0000000100dfb0a0 specialized specialized StaticString.withUTF8Buffer<A> ((UnsafeBufferPointer<UInt8>) -> A) -> A + 355
4    libswiftCore.dylib                 0x0000000100fb32b0 partial apply for (_assertionFailed(StaticString, StaticString, StaticString, UInt, flags : UInt32) -> Never).(closure #1).(closure #1) + 144
5    libswiftCore.dylib                 0x0000000100dfb5b0 specialized specialized String._withUnsafeBufferPointerToUTF8<A> ((UnsafeBufferPointer<UInt8>) throws -> A) throws -> A + 124
6    libswiftCore.dylib                 0x0000000100f57af0 partial apply for (_assertionFailed(StaticString, String, StaticString, UInt, flags : UInt32) -> Never).(closure #1) + 185
7    libswiftCore.dylib                 0x0000000100dfb0a0 specialized specialized StaticString.withUTF8Buffer<A> ((UnsafeBufferPointer<UInt8>) -> A) -> A + 355
8    libswiftCore.dylib                 0x0000000100dfae80 _assertionFailed(StaticString, String, StaticString, UInt, flags : UInt32) -> Never + 144
9    libswiftCore.dylib                 0x0000000100e1e540 swift_unexpectedError_merged + 569
10   App                                0x0000000100001ef0 main + 2798
11   libdyld.dylib                      0x00007fff974375ac start + 1
Program ended with exit code: 9

I am using the VaporPostgreSQL package. Here is my Package.swift:

import PackageDescription

let package = Package(
    name: "mist",
    dependencies: [
        .Package(url: "https://github.com/vapor/vapor.git", majorVersion: 1, minor: 2),
        .Package(url: "https://github.com/vapor/postgresql-provider.git", majorVersion: 1, minor: 1)
    ],
    exclude: [
        "Config",
        "Database",
        "Localization",
        "Public",
        "Resources",
        "Tests",
    ]
)

And main.swift:

import Vapor
import VaporPostgreSQL
import Auth
import HTTP

let drop = Droplet()
let auth = AuthMiddleware(user: User.self)

try drop.addProvider(VaporPostgreSQL.Provider.self)
drop.preparations.append(Post.self)
drop.preparations.append(User.self)
drop.preparations.append(Site.self)
drop.middleware.append(auth)

let admin = AdminController()
var site = Site(name: "", theme: "")

if let retreivedSite = try Site.all().first {
    site = retreivedSite
} else {
    drop.get { req in
        return Response(redirect: "http://localhost:8080/login")
    }
}

drop.get { req in
    return try drop.view.make("Themes/VaporDark/index", [
        "posts": Node(node: JSON(Post.all().makeNode()))
    ])
}

admin.addRoutes(to: drop)

drop.resource("posts", PostController())

drop.run()

My postgres version is 9.6.1

For some reason VaporPostgreSQL won't update and I think that might be part of the problem. I have tried vapor xcode, vapor build and vapor clean, but I can't get the latest version.


Answer:

I think the issue is here:

if let retreivedSite = try Site.all().first {
    site = retreivedSite
} else {
    drop.get { req in
        return Response(redirect: "http://localhost:8080/login")
    }
}

More specifically, the Site.all() call. We don't prepare the models until the run() command is called, so, to look up Site before that point, the model will need to be prepared manually.

Hope this helps!

Question:

I would like to have a table with a string column as a primary key without having to use raw SQL syntax.

Here's my fluent "preparation":

static func prepare(_ database: Database) throws {    
    try database.create("roles") { roles in
        roles.id("name")
        roles.string("readable_name")
    }
}

According to both my tests and the docs, resulting query will be similar to:

CREATE TABLE `roles` (`name` INTEGER PRIMARY KEY NOT NULL, `readable_name` TEXT NOT NULL)

I could not, so far, find a way to have a string (TEXT, VARCHAR, ...) as a primary key without raw SQL syntax and i would like to know whether it's possible to do it or not using the fluent query builder which comes with vapor.


Answer:

Support for ID types besides INT was added in Fluent 2.

https://docs.vapor.codes/2.0/fluent/model/#id-type

Question:

I have built a MySQL database with multiple tables and complex relationships, but when I go through the vapor documentation, specifically, in the building the model phase, there is a method for creating the table (that my model class will interact with).

static func prepare(_ database: Database) throws {
    try database.create("users") { users in
        users.id()
        users.string("name")
    }
}

However, I don't want to use it because the table that I already have contain foreign keys and types like DATETIME (which I don't know how to declare within the swift context.) is there a way to link my already built tables with vapor?


Answer:

This is somewhere Vapor (or more correctly Fluent, which is the database level of Vapor) is a bit limited.

Yes, you can use your existing tables. In your prepare(_:) method, you can simply leave the implementation empty without creating a table at all. You should also leave revert(_:) empty as well.

In your init(node:in:) initialiser and makeNode(context:) method, you will need to map between the column names and types in your table and the property types in your Swift model.

Question:

The Vapor 3 documentation doesn't say much about database configuration other than to "register a DatabasesConfig struct to your services."

Tutorials (such as this one) suggest that you implement the configuration in the App/configure.swift file like this:

let mysqlConfig = MySQLDatabaseConfig(
    hostname: "127.0.0.1",
    port: 3306,
    username: "root",
    password: "root",
    database: "mycooldb"
)
services.register(mysqlConfig)

But my configure.swift file is being tracked by git, and I don't want to commit my username and password.

How do I supply an external configuration file for handling the database connection?

It appears that earlier versions of Vapor used JSON configuration files. Is this functionality completely gone? I can't find any mention of it in the current documentation.


Answer:

The most popular way to do this is using environment variables. You can set them in the Xcode scheme or the terminal:

export DB_PASSWORD=root

Then get it in your configuration:

guard let password = Environment.get("DB_PASSWORD") else {
    throw Abort(.internalServerError)
}

Question:

While doing Ray Wenderlich tutorial "Server Side Swift with Vapor: Persisting Models" I tried to add one more parameter(param) to the class Acronyms.

import Vapor

final class Acronym: Model {

  var id: Node?
  var exists: Bool = false

  var short: String
  var long: String
  var param: String

    init(short: String, long: String, param: String) {
    self.id = nil
    self.short = short
    self.long = long
    self.param = param
  }

  init(node: Node, in context: Context) throws {
    id = try node.extract("id")
    short = try node.extract("short")
    long = try node.extract("long")
    param = try node.extract("param")
  }

  func makeNode(context: Context) throws -> Node {
    return try Node(node: [
      "id": id,
      "short": short,
      "long": long,
      "param": param
    ])
  }

  static func prepare(_ database: Database) throws {
    try database.create("acronyms") { users in
      users.id()
      users.string("short")
      users.string("long")
      users.string("param")
    }
  }

  static func revert(_ database: Database) throws {
    try database.delete("acronyms")
  }

}

At first I run this code without one more parameter. And it works. But when i added one it fails.

Error: 500The operation couldn’t be completed. (PostgreSQL.DatabaseError error 1.)

My main.swift:

import Vapor
import VaporPostgreSQL

let drop = Droplet(
    preparations: [Acronym.self],
    providers: [VaporPostgreSQL.Provider.self]
)

drop.get("hello") { request in
    return "Hello, world!"
}

drop.get("version") { req in
    if let db = drop.database?.driver as? PostgreSQLDriver {
        let version = try db.raw("SELECT  version()")
        return try JSON(node: version)
    } else {
        return "No db connection"
    }
}

drop.get("test") { request in
    var acronym = Acronym(short: "AFK", long: "Away From Keyboard", param: "One More Parametr")
    try acronym.save()
    return try JSON(node: Acronym.all().makeNode())
}

drop.run()

Answer:

I assume you didn't revert the database. You changed the model's properties, so just write in terminal vapor run prepare --revert . That will revert your database and vapor will be able to create new parameter.

Question:

I'm trying to setup a Vapor 3 project with SQLite. In the configure.swift file, I have the following setup related to sqlite:

try services.register(FluentSQLiteProvider())

...

// Get the root directory of the project
// I have verified that the file is present at this path
let path = DirectoryConfig.detect().workDir + "db_name.db"
let sqlite: SQLiteDatabase
do {
    sqlite = try SQLiteDatabase(storage: .file(path: path))
    print("connected") // called
} catch {
    print(error) // not called
    return
}

var databases = DatabasesConfig()
databases.add(database: sqlite, as: .sqlite)
services.register(databases)

In the database, I have a table called posts, that I want to query and return all entries from:

This is the Post implementation inside /Sources/App/Models/:

final class Post: Content {
    var id: Int?
    var title: String
    var body: String

    init(id: Int? = nil, title: String, body: String) {
        self.id = id
        self.title = title
        self.body = body
    }
}
extension Post: SQLiteModel, Migration, Parameter { }

I have also added the migration in configure.swift:

var migrations = MigrationConfig()
migrations.add(model: Post.self, database: .sqlite)
services.register(migrations)

In routes.swift, I define the posts route as follows:

router.get("posts") { req in
    return Post.query(on: req).all()
}

Now, when calling localhost:8080/posts, I get:

[]

Have I not properly connected the database? Have I missed something?


Answer:

It seems like the table name generated by fluent is different than your db table name as Fluent generate the table with name same as the Model class name. In your case Post.

Add static property entity in your model class Post to define a custom table name.

like this:

public static var entity: String {
    return "posts"
}

Question:

I'm trying to build APIs using Swift and I've chosen to use Vapor.

I've created a SQLite database and am able to connect to it using a DB client.

Now I want my Swift Vapor project to connect to it as well using the FluentSQLite package.

I've created my database in the root folder of my project:

/Users/rutgerhuijsmans/Documents/runk-3.0

My database is called runk-3.0-database

The folder looks like this:

I try to connect to my DB using the following configuration:

import FluentSQLite
import Vapor

/// Called before your application initializes.
public func configure(_ config: inout Config, _ env: inout Environment, _ services: inout Services) throws {
    /// Register providers first
    try services.register(FluentSQLiteProvider())

    /// Register routes to the router
    let router = EngineRouter.default()
    try routes(router)
    services.register(router, as: Router.self)

    /// Register middleware
    var middlewares = MiddlewareConfig() // Create _empty_ middleware config
    /// middlewares.use(FileMiddleware.self) // Serves files from `Public/` directory
    middlewares.use(ErrorMiddleware.self) // Catches errors and converts to HTTP response
    services.register(middlewares)

    let sqlite: SQLiteDatabase?
    do {
        sqlite = try SQLiteDatabase(storage: .file(path: "runk-3.0-database"))
        print("data base connected") // This gets printed

        /// Register the configured SQLite database to the database config.
        var databases = DatabasesConfig()
        databases.add(database: sqlite!, as: .sqlite)
        services.register(databases)

        /// Configure migrations
        var migrations = MigrationConfig()
        migrations.add(model: User.self, database: .sqlite)
        services.register(migrations)
    } catch {
        print("couldn't connect") // This doesn't get printed
    }
}

What am I doing wrong?


Answer:

As IMike17 explained, your code just creates the new DB file into the Build/Products/Debug or release folder. You have to set full path dynamically as below:

do {
let directory = DirectoryConfig.detect()
let filePath = directory.workDir + "runk-3.0-database"
sqlite = try SQLiteDatabase(storage: .file(path: filePath)) 
......

Question:

I am writing a server using Swift 4 + Vapor framework, Fluent ORM and PostgreSQL as a driver. I've got a User Model which should have subscribers and subscriptions (which are also User Models). I have two options here: 1. store arrays with unique ids of subscriptions/subscribers or 2. build a one-to-many User-User relation. Which one do you think is better and how can I impemplement it?


Answer:

Storing an array is not optimal. Querying your database to find all a User's subscribers will require parsing every User's subscriptions array and finding those which contain your target User's ID. A relation is a better idea.

Fluent uses the Pivot class to model many-to-many relations. Because it's a self-referencing relation, to avoid ID key conflicts you will probably find it easiest to create your own 'through' model.

import FluentProvider
import Vapor

final class Subscription: Model, PivotProtocol {

  typealias Left = User
  typealias Right = User

  var subscriberId: Identifier
  var subscribedId: Identifier

  init(
    subscriberId: Identifier,
    subscribedId: Identifier
  ) {
    self.subscriberId = subscriberId
    self.subscribedId = subscribedId
  }

  let storage = Storage()

  static let leftIdKey = "subscriber_id"
  static let rightIdKey = "subscribed_id"

  init(row: Row) throws {
    subscriberId = try row.get("subscriber_id")
    subscribedId = try row.get("subscribed_id")
  }

  func makeRow() throws -> Row {
    var row = Row()
    try row.set("subscriber_id", subscriberId)
    try row.set("subscribed_id", subscribedId)
    return row
  }

}

extension User {
  var subscribers: Siblings<User, User, Subscription> {
    return siblings(localIdKey: "subscriber_id", foreignIdKey: "subscribed_id")
  }
  var subscribed: Siblings<User, User, Subscription> {
    return siblings(localIdKey: "subscribed_id", foreignIdKey: "subscriber_id")
  }
}

Question:

Is this possible to use in-memory FluentSQLite provider for testing purpose and FluentPostgreSQL for app's models?


Answer:

It depends....

In short for simple apps yeah you can. You basically need to make your models generic and then set up the generic models from your configuration all the way down. See how the benchmark models are set up here.

In reality - no you can't. As soon as you want to do anything that isn't standard (TEXT column type) etc, you need to make your models specific to the DB type.

The way to do it is to use the repository pattern and completely abstract away your database from your application logic. See the Vapor style guide for more details.

Question:

I'm trying to create a user and access token record in my database.

However I can't figure out how to do this.

My code looks like this:

// Create new user
func create(_ req: Request) throws -> Future<AccessToken> {
    return try req.content.decode(User.self).flatMap { user in
        user.pushToken = ""
        user.create(on: req).map {_ -> EventLoopFuture<AccessToken> in
            let accessToken = AccessToken(accessToken: UUID().uuidString, userID: user.id!)
            return accessToken.create(on: req)
        }
    }
}

I create a user (this works well) then I want to create an access token tied to that user (through user ID)

Because of this I need to know the user ID of the user that I just created. However this code doesn't seem to compile.

Xcode is giving me: Missing return in a closer expected to return EventLoopFuture<AccessToken>


Answer:

Missing a return in the user.create(on: req).map {_ -> EventLoopFuture<AccessToken> in ?

Question:

I'm just getting started with programming and vapor and have the following querying problem.

When a user makes a request to /api/trip/new to create a new trip, I want to retrieve the first driver from the database and assign their ID to a Trip object. Then I want to save both the trip and trip request in the database.

I have the following code:

func newTripHandler(_ req: Request, data: TripRequestCreateData) throws -> Future<TripRequest> {
let passenger = try req.requireAuthenticated(Passenger.self)
let tripRequest = try TripRequest(pickupLat: data.pickupLat,
                                  pickupLng: data.pickupLng,
                                  destLat: data.destLat,
                                  destLng: data.destLng,
                                  pickupAddress: data.pickupAddress,
                                  destAddress: data.destAddress,
                                  passengerId: passenger.requireID())

//Passenger's location
let lat = tripRequest.pickupLat
let lng = tripRequest.pickupLng

//Passenger's destination
let toLat = tripRequest.destLat
let toLng = tripRequest.destLng

let trip = Trip(passengerId: tripRequest.passengerId!,
                pickupLat: lat,
                pickupLng: lng,
                destLat: toLat,
                destLng: toLng,
                pickupAddress: tripRequest.pickupAddress,
                destAddress: tripRequest.destAddress)

let driver = Driver.query(on: req).first().unwrap(or: Abort(.notFound))

let driverId = driver.map(to: Driver.self) { driver in
  return driver.id
}

trip.driverId = driverId

print(trip)

return tripRequest.save(on: req)
}

struct TripRequestCreateData: Content {
  let pickupLat: Double
  let pickupLng: Double
  let destLat: Double
  let destLng: Double
  let pickupAddress: String
  let destAddress: String
}

I’m not sure how to save two models and how to properly retrieve the driver from the database. The driverId constant is of type EventLoopFuture instead of Driver so I can't assign the ID to trip's driverId property.

How can I adjust my code to achieve this?


Answer:

When working with the database, you only get Futures. (More about Futures in The Vapor Docs.)

So to get the driver, you already used the correct code, but have to go on inside the map codeblock. When returning a Future inside the map, use flatMap, so you don't have a Futures inside Futures.

Try this:

return Driver.query(on: req).first().unwrap(or: Abort(.notFound)).flatMap { driver in
    trip.driverId = driver.id

    return trip.save(on: req).flatMap { trip in
        return tripRequest.save(on: req)
    }
}

(I chose this way to show how Futures work - there are some ways to make this code cleaner, e.g. chaining Futures or using flatten.)

Question:

// This is a continuation of the questions I have asked from a tutorial by Paul Hudson on youtube -

I have tried to add items to a database (see image below) -

When I should click on the "Add" button on the image above, the boxes should become EMPTY (See image below). Though .Quantum Pizza will not be added to the list of .Statin Island Pizza and .Country pizza, because I have not done further coding), but it should be as the image below -

but, the result is as follows -

Now, I am posting the codes -----

configure.swift -

import  Fluent
import FluentSQLite
import Vapor
import Leaf // added

public func configure(_ config: inout Config, _ env: inout Environment, _ services: inout Services) throws {
    // Register routes to the router
    let router = EngineRouter.default()
    try routes(router)
    services.register(router, as: Router.self)

    let leafProvider = LeafProvider()    // added
    try services.register(leafProvider)  // added
    config.prefer(LeafRenderer.self, for: ViewRenderer.self)// added

    let directoryConfig = DirectoryConfig.detect()
    services.register(directoryConfig)
    try services.register(FluentSQLiteProvider())
    var databaseConfig = DatabasesConfig()
    let db = try SQLiteDatabase(storage: .file(path:"\(directoryConfig.workDir)pizza.db"))

    databaseConfig.add(database: db, as: .sqlite)
    services.register(databaseConfig)

    var migrationConfig = MigrationConfig()
    migrationConfig.add(model: Pizza.self, database: .sqlite)
    services.register(migrationConfig)
    let serverConfigure = NIOServerConfig.default(hostname: "0.0.0.0", port: 9090)
    services.register(serverConfigure)
}

routes.swift -

import Routing
import Vapor
import FluentSQLite

public func routes(_ router: Router) throws {
    router.get { req -> Future <View> in
        let Newyorker = Pizza(id: 5, name: "Statin Island Pizza", description: "Impractical Jokers Funny Pizza", price: 55)
        let Traditional = Pizza(id: 5, name: "Country Pizza ", description: "Johny Cash Special", price: 55)

        return try req.view().render("welcome",["pizza":[Newyorker,Traditional]])
    }

    router.post(Pizza.self, at: "add") { req, pizza -> Future<Response> in
        return pizza.save(on:req).map(to:Response.self) { Pizza in
            return req.redirect(to: "/")
        }
    }
}

pizza.swift -

import Foundation
import Vapor
import FluentSQLite

struct Pizza: Encodable, Content, Decodable, SQLiteModel, Migration {
    var id:  Int?
    var name: String
    var description: String
    var price: Int
}

leaf screenshot (I tried to paste code, but couldn't, in the correct format. So adding screeshot) -

Edit 1: screenshot after I click on the Add button -

Ill be happy to provide you any further information if you need. Also, I would like to know if the title of my question should be modfied or anyhing should be added to it. Thank you.


Answer:

Your forms action should be action="add" (you're missing the closing quotation to close the action)