Understanding NoSQL Databases in Go

What is NoSQL?

NoSQL databases provide flexible, scalable data storage solutions that differ from traditional relational databases in several key ways:

1. Schema Flexibility:

   • No fixed table structure
   • Dynamic field addition/removal
   • Varied data formats (documents, key-value pairs, etc.)

2. Scalability:

   • Horizontal scaling across multiple servers
   • Built-in sharding capabilities
   • Distributed architecture support

3. Performance:

   • Optimized for specific data models
   • Fast read/write operations
   • Efficient handling of large data volumes

Types of NoSQL Databases

1. Document Stores (MongoDB)

• Store data in JSON-like documents
• Support nested data structures
• Flexible schema design
• Good for: Content management, catalogs, user profiles

2. Key-Value Stores (Redis)

• Simple key-value pair storage
• Extremely fast operations
• In-memory with persistence options
• Good for: Caching, session management, real-time analytics

3. Wide-Column Stores (Cassandra)

• Column-family based storage
• Highly scalable
• Optimized for large datasets
• Good for: Time-series data, IoT applications

4. Graph Databases (Neo4j)

• Store interconnected data
• Optimize relationship queries
• Natural fit for network structures
• Good for: Social networks, recommendation engines

Working with MongoDB

Understanding MongoDB Concepts

1. Documents and Collections:

   • Documents are JSON-like records
   • Collections group similar documents
   • No enforced schema requirements

2. ObjectID:

   • Unique identifier for documents
   • Automatically generated
   • Contains timestamp information

3. Indexes:

   • Improve query performance
   • Support various types (single, compound, text)
   • Can be unique or sparse

Here's how to work with MongoDB in Go:

package main

import (
    "context"
    "log"
    "time"
    
    "go.mongodb.org/mongo-driver/mongo"
    "go.mongodb.org/mongo-driver/mongo/options"
)

type MongoDB struct {
    client   *mongo.Client
    database string
}

func NewMongoDB(uri, database string) (*MongoDB, error) {
    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()
    
    client, err := mongo.Connect(ctx, options.Client().ApplyURI(uri))
    if err != nil {
        return nil, err
    }
    
    // Ping database
    if err := client.Ping(ctx, nil); err != nil {
        return nil, err
    }
    
    return &MongoDB{
        client:   client,
        database: database,
    }, nil
}

Document Operations

Understanding CRUD operations in MongoDB:

1. Create (Insert):

   • Single document insertion
   • Bulk insertions
   • Insert with options (ordered/unordered)

2. Read (Find):

   • Query filters
   • Projections
   • Sorting and limiting

3. Update:

   • Update operators ($set, $inc, etc.)
   • Atomic operations
   • Upsert capability

4. Delete:

   • Single document removal
   • Bulk deletions
   • Soft delete patterns

Example implementation:

type User struct {
    ID        primitive.ObjectID `bson:"_id,omitempty"`
    Name      string            `bson:"name"`
    Email     string            `bson:"email"`
    CreatedAt time.Time         `bson:"created_at"`
    UpdatedAt time.Time         `bson:"updated_at"`
}

// Create operation
func (db *MongoDB) CreateUser(ctx context.Context, user *User) error {
    collection := db.client.Database(db.database).Collection("users")
    
    user.CreatedAt = time.Now()
    user.UpdatedAt = time.Now()
    
    result, err := collection.InsertOne(ctx, user)
    if err != nil {
        return err
    }
    
    user.ID = result.InsertedID.(primitive.ObjectID)
    return nil
}

// Read operation
func (db *MongoDB) GetUser(ctx context.Context, id primitive.ObjectID) (*User, error) {
    collection := db.client.Database(db.database).Collection("users")
    
    var user User
    err := collection.FindOne(ctx, bson.M{"_id": id}).Decode(&user)
    if err != nil {
        if err == mongo.ErrNoDocuments {
            return nil, fmt.Errorf("user not found: %s", id)
        }
        return nil, err
    }
    
    return &user, nil
}

Working with Redis

Understanding Redis Concepts

1. Data Structures:

   • Strings: Simple key-value pairs
   • Lists: Ordered collections
   • Sets: Unique unordered elements
   • Hashes: Field-value pairs
   • Sorted Sets: Scored ordered elements

2. Persistence:

   • RDB snapshots
   • AOF log
   • Hybrid persistence

3. Expiration:

   • TTL (Time To Live)
   • Automatic cleanup
   • Key eviction policies

Basic Redis setup in Go:

type Redis struct {
    client *redis.Client
}

func NewRedis(addr string) (*Redis, error) {
    client := redis.NewClient(&redis.Options{
        Addr:     addr,
        Password: "", // no password set
        DB:       0,  // use default DB
    })
    
    // Test connection
    ctx := context.Background()
    if err := client.Ping(ctx).Err(); err != nil {
        return nil, err
    }
    
    return &Redis{client: client}, nil
}

Common Redis Patterns

1. Caching Layer:

   • Cache frequently accessed data
   • Handle cache invalidation
   • Implement cache-aside pattern

2. Session Storage:

   • Store session data
   • Handle expiration
   • Manage concurrent access

3. Rate Limiting:

   • Track request counts
   • Implement sliding windows
   • Handle distributed rate limiting

Example implementations:

// Caching implementation
type Cache struct {
    redis  *Redis
    prefix string
}

func (c *Cache) Get(ctx context.Context, key string, value interface{}) error {
    return c.redis.Get(ctx, c.CacheKey(key), value)
}

func (c *Cache) Set(ctx context.Context, key string, value interface{}, expiration time.Duration) error {
    return c.redis.Set(ctx, c.CacheKey(key), value, expiration)
}

// Rate limiting implementation
type RateLimiter struct {
    redis     *Redis
    key       string
    limit     int
    window    time.Duration
}

func (rl *RateLimiter) Allow(ctx context.Context, identifier string) (bool, error) {
    key := fmt.Sprintf("%s:%s", rl.key, identifier)
    
    pipe := rl.redis.client.Pipeline()
    pipe.Incr(ctx, key)
    pipe.Expire(ctx, key, rl.window)
    
    results, err := pipe.Exec(ctx)
    if err != nil {
        return false, err
    }
    
    count := results[0].(*redis.IntCmd).Val()
    return count <= int64(rl.limit), nil
}

Best Practices

1. Data Modeling

• Choose appropriate data structures
• Plan for query patterns
• Consider data access patterns
• Design for scalability

2. Performance Optimization

• Use appropriate indexes
• Implement caching strategies
• Monitor query performance
• Handle connection pooling

3. Error Handling

• Implement retry mechanisms
• Handle timeout scenarios
• Manage connection failures
• Log errors appropriately

Common Challenges

1. Consistency:

   • Handle eventual consistency
   • Implement optimistic locking
   • Manage concurrent updates

2. Scaling:

   • Plan for horizontal scaling
   • Handle data distribution
   • Manage connection pools

3. Monitoring:

   • Track performance metrics
   • Monitor resource usage
   • Set up alerting

Next Steps

• Learn about Query Building
• Explore Transactions
• Study Connection Pooling