Platform

 View Only

Bill and the On-Prem MongoDB Data

By Guillermo Cuadrado posted Thu January 09, 2025 07:59 AM

  

How to allocate database costs to consuming applications? Often times, TBMAs need to import external data into the TBM engine to define cost allocation strategies based on consumption. In this example, Bill works with an application owner to define an allocation strategy for MongoDB database costs.

***

"Hey, Bill. I'd like to talk about MongoDB costs." Said Amy Rose, the owner of the online retail application.


"How can I help, Amy?" Bill said.


"I'd like to have a view on database costs for my application, beyond the current flat fee allocation we have."

"Sure. Let me pull Michelle into this call." Michelle Cooper was the database administrator for MongoDB.

"Hey, Michelle. What consumption data can we get from MongoDB, so that we can allocate the associated costs to consuming applications?" Bill said.

"MongoDB provides a utility—mongoexport—that extracts data from the database, but I suppose that's not what you're after." Michelle said.

"What we need is figuring out how large the database for each application is. That is usually a defensible allocation strategy." Bill said. "Something like this would be fine."

Application Database Size/Storage (MB)
App1 db1                                  848
App2 db2                                  695
App3 db3                                    30
App4 db4                                  385
App5 db5                                  928
App6 db6                                  344
App7 db7                                  515

Fig. 1-Database size by Application

"If we had that data in a CSV file, we could reasonably easily load it to our TBM system and derive the cost allocation percentages, based on the relative weights," Bill said.

"What if we add another application?" Amy said. "Wouldn't the unit cost change?"

"One option would be relating those weights to the total maximum capacity, not just the relative weights. Would that be an option, Michelle." Bill asked.

"I'll have to look into that, but I believe so," Michelle said. "I can write a Python script that will list all the databases and interrogate each one by one, to get their storage size. Something like this:"

from pymongo import MongoClient
# Connect to MongoDB
client = MongoClient('mongodb://localhost:27017/')

# List all databases
databases = client.list_database_names()

# Function to get the size of a database
def get_db_size(database_name):
    db = client[database_name]
    stats = db.command("dbstats")
    return {
        'dataSize': stats['dataSize'],
        'storageSize': stats['storageSize'],
        'indexSize': stats['indexSize']
    }

# Iterate over each database and get its size
db_sizes = {}
for db_name in databases:
    db_sizes[db_name] = get_db_size(db_name)

"Then we can store the databases and respective size as a CSV file."

 "Sure, that's exactly what we need. Michelle." Bill said. "If you place that CSV file in a folder I can access, I will set up a Datalink File System connector to pick it up and ship it to our engine."

"Fine, just send me a work order and I'll write the program and have it run periodically. You can then run your automation and pick the data whenever you want," Michelle said.

"Yes, in the work order, I'll give you a network address where you can place the file." Said Bill. "Amy, we'll need something else to finalize the allocation."

"What's that?"

"A list linking applications to database names. With that, we can close the loop and define the allocation strategy. I'll add a column called 'Database Size' and store the data we pick up from the CSV file there. That will serve as the allocation weight."

"Sure, Bill. I'll put together that list and send it to you."

***

#BillTheTBMGuy

Copyright (c) Guillermo Cuadrado 2025

1 comment
15 views

Permalink

Comments

Fri January 10, 2025 06:45 AM