Transactions and Write Management

DB plugin provides a robust transaction system for managing write operations. Transactions ensure data consistency, enable batch operations, and provide validation before any data reaches the database.

Transaction Lifecycle

Basic Transaction Pattern

beginWrites / commitWrites / endWrites

1
Database db = Database.Static.getDefault();
2

3
db.beginWrites();
4
try {
5
    // Make changes
6
    article1.save();
7
    article2.save();
8
    author.save();
9

10
    // Commit changes
11
    db.commitWrites();
12

13
} finally {
14
    // Always clean up
15
    db.endWrites();
16
}

What Happens

1. beginWrites() - Starts a transaction, increments depth counter
2. save() - Buffers the write, doesn't touch the database yet
3. commitWrites() - Validates all buffered writes, then writes them atomically
4. endWrites() - Decrements depth counter, cleans up resources

Why Use Transactions

1
// Without transaction - NOT atomic
2
article.save();  // Might succeed
3
author.save();   // Might fail
4
// Now you have inconsistent data!
5

6
// With transaction - atomic
7
Database db = Database.Static.getDefault();
8
db.beginWrites();
9
try {
10
    article.save();
11
    author.save();
12
    db.commitWrites();  // Both succeed or both fail
13
} finally {
14
    db.endWrites();
15
}

Nested Transactions

DB plugin supports nested transactions using a depth counter:

1
Database db = Database.Static.getDefault();
2

3
db.beginWrites();  // Depth 1
4
try {
5
    article.save();
6

7
    db.beginWrites();  // Depth 2 (nested)
8
    try {
9
        author.save();
10
        db.commitWrites();  // Commits only at depth 1
11
    } finally {
12
        db.endWrites();  // Back to depth 1
13
    }
14

15
    db.commitWrites();  // Actually commits everything
16
} finally {
17
    db.endWrites();  // Depth 0
18
}

How Nesting Works

• beginWrites() increments a counter
• endWrites() decrements it
• commitWrites() only writes when counter reaches 0
• Allows method composition without worrying about transaction state

1
Database db = Database.Static.getDefault();
2

3
// Called standalone - commits immediately
4
saveArticleWithAuthor(article1, author1);
5

6
// Called within transaction - commits with outer transaction
7
db.beginWrites();
8
try {
9
    saveArticleWithAuthor(article1, author1);
10
    saveArticleWithAuthor(article2, author2);
11
    db.commitWrites();  // All 4 objects committed together
12
} finally {
13
    db.endWrites();
14
}

Commit Strategies

commitWrites - Immediate Consistency

Commits changes immediately with strong consistency guarantees:

1
Database db = Database.Static.getDefault();
2

3
db.beginWrites();
4
try {
5
    article.save();
6
    db.commitWrites();
7
    // Data is immediately visible to other transactions
8
} finally {
9
    db.endWrites();
10
}

Use when:

• You need immediate read-after-write consistency
• Other processes depend on this data immediately
• Working with critical data (financial, inventory, etc.)

commitWritesEventually - Eventual Consistency

Commits changes for eventual consistency, allowing for optimization:

1
Database db = Database.Static.getDefault();
2

3
db.beginWrites();
4
try {
5
    article.save();
6
    db.commitWritesEventually();
7
    // Data will be visible soon, but maybe not immediately
8
} finally {
9
    db.endWrites();
10
}

Use when:

• Eventual consistency is acceptable
• Performance is critical
• Working with caching or replication systems
• Bulk imports or batch operations

The database implementation may:

• Batch writes together
• Delay replication
• Optimize index updates
• Trade immediate consistency for throughput

Isolated Writes

Create an isolated write session that doesn't interact with the main transaction:

1
Database db = Database.Static.getDefault();
2

3
// Main transaction
4
db.beginWrites();
5
try {
6
    article.save();
7

8
    // Isolated session
9
    db.beginIsolatedWrites();
10
    try {
11
        // This saves independently
12
        auditLog.save();
13
        db.commitWrites();
14
    } finally {
15
        db.endWrites();
16
    }
17

18
    // Main transaction continues
19
    db.commitWrites();
20
} finally {
21
    db.endWrites();
22
}

Use cases:

• Audit logging that should succeed even if main transaction fails
• Separate concerns (e.g., metrics, analytics)
• Writing to different databases

Write Buffering and Validation

The Validation Phase

All validation happens before any database writes:

1
Database db = Database.Static.getDefault();
2

3
db.beginWrites();
4
try {
5
    // These are buffered
6
    article1.save();
7
    article2.save();
8
    article3.save();
9

10
    // This triggers validation of ALL three
11
    db.commitWrites();
12

13
    // If any validation fails, NONE are written
14
} catch (ValidationException e) {
15
    // Handle validation errors for all objects
16
    List<State> statesWithErrors = e.getStates();
17
} finally {
18
    db.endWrites();
19
}

Validation Order

1. Annotation validation - @Required, @Maximum, etc.
2. beforeSave() hooks - Custom pre-save logic
3. onValidate() hooks - Custom validation logic
4. afterValidate() hooks - Post-validation logic
5. Database write - Only if all validation passes
6. afterSave() hooks - Post-save logic

Example: Custom Validation

1
public class ArticleWithValidation extends Record {
2

3
    @Required
4
    private String title;
5

6
    @Required
7
    private String content;
8

9
    @Override
10
    protected void beforeSave() {
11
        // Normalize data before validation
12
        if (title != null) {
13
            title = title.trim();
14
        }
15
    }
16

17
    @Override
18
    protected void onValidate() {
19
        // Custom validation logic
20
        if (content != null && content.length() < 100) {
21
            getState().addError(
22
                getState().getField("content"),
23
                new IllegalArgumentException("Content must be at least 100 characters")
24
            );
25
        }
26

27
        if (title != null && title.toLowerCase().contains("spam")) {
28
            getState().addError(
29
                getState().getField("title"),
30
                new IllegalArgumentException("Title contains forbidden words")
31
            );
32
        }
33
    }
34

35
    @Override
36
    protected void afterSave() {
37
        // Post-save logic (cache invalidation, notifications, etc.)
38
        System.out.println("Article saved: " + getLabel());
39
    }
40

41
    // Getters and setters
42
    public String getTitle() {
43
        return title;
44
    }
45

46
    public void setTitle(String title) {
47
        this.title = title;
48
    }
49

50
    public String getContent() {
51
        return content;
52
    }
53

54
    public void setContent(String content) {
55
        this.content = content;
56
    }
57
}

Error Handling

Handling ValidationException

1
Database db = Database.Static.getDefault();
2

3
db.beginWrites();
4
try {
5
    article.save();
6
    db.commitWrites();
7

8
} catch (ValidationException e) {
9
    // Get all validation errors
10
    for (State state : e.getStates()) {
11
        System.out.println("Errors in " + state.getType().getInternalName() + ":");
12

13
        for (ObjectField field : state.getErrorFields()) {
14
            String fieldName = field.getInternalName();
15
            for (String error : state.getErrors(field)) {
16
                System.out.println("  " + fieldName + ": " + error);
17
            }
18
        }
19
    }
20

21
} catch (DatabaseException e) {
22
    // Handle database errors (connection, constraint violations, etc.)
23
    System.err.println("Database error: " + e.getMessage());
24
    throw e;
25

26
} finally {
27
    db.endWrites();
28
}

Handling Database Errors

1
Database db = Database.Static.getDefault();
2

3
db.beginWrites();
4
try {
5
    article.save();
6
    db.commitWrites();
7

8
} catch (DatabaseException e) {
9
    // Check for specific error types
10
    if (e.getCause() instanceof SQLException) {
11
        SQLException sqlEx = (SQLException) e.getCause();
12

13
        // Unique constraint violation
14
        if (sqlEx.getErrorCode() == 1062) {  // MySQL duplicate entry
15
            throw new RuntimeException("Article already exists", e);
16
        }
17

18
        // Deadlock
19
        if (sqlEx.getErrorCode() == 1213) {  // MySQL deadlock
20
            // Retry logic
21
            // return retryOperation();
22
        }
23
    }
24

25
    throw e;
26

27
} finally {
28
    db.endWrites();
29
}

Automatic Retry

DB plugin automatically retries certain recoverable errors:

1
// This is handled automatically by AbstractDatabase
2
Database db = Database.Static.getDefault();
3

4
db.beginWrites();
5
try {
6
    article.save();
7
    db.commitWrites();  // Will retry on transient errors
8
} finally {
9
    db.endWrites();
10
}

Automatically retried errors:

• Connection timeouts
• Deadlocks
• Transient network failures
• Lock wait timeouts

Uses exponential backoff with configurable max attempts.

Batch Operations

Efficient Batch Writes

1
Database db = Database.Static.getDefault();
2

3
db.beginWrites();
4
try {
5
    for (ArticleData data : dataList) {
6
        Article article = new Article();
7
        article.setTitle(data.getTitle());
8
        article.setContent(data.getContent());
9
        article.setAuthor(data.getAuthor());
10
        article.save();  // Buffered, not written yet
11
    }
12

13
    // All articles validated and written in one batch
14
    db.commitWrites();
15

16
    System.out.println("Imported " + dataList.size() + " articles");
17

18
} catch (ValidationException e) {
19
    System.err.println("Import failed due to validation errors");
20
    // None of the articles are saved
21
    throw e;
22

23
} finally {
24
    db.endWrites();
25
}

Batch with Progress Tracking

1
Database db = Database.Static.getDefault();
2
int batchSize = 100;
3
int processed = 0;
4

5
for (int i = 0; i < dataList.size(); i += batchSize) {
6
    int end = Math.min(i + batchSize, dataList.size());
7
    List<ArticleData> batch = dataList.subList(i, end);
8

9
    db.beginWrites();
10
    try {
11
        for (ArticleData data : batch) {
12
            Article article = new Article();
13
            // ... populate article
14
            article.save();
15
        }
16

17
        db.commitWrites();
18
        processed += batch.size();
19
        System.out.println("Processed " + processed + " / " + dataList.size());
20

21
    } catch (Exception e) {
22
        System.err.println("Batch failed at index " + i);
23
        throw e;
24

25
    } finally {
26
        db.endWrites();
27
    }
28
}

Parallel Batch Processing

1
int numThreads = Runtime.getRuntime().availableProcessors();
2
ExecutorService executor = Executors.newFixedThreadPool(numThreads);
3
int batchSize = 100;
4

5
List<Future<?>> futures = new ArrayList<>();
6

7
for (int i = 0; i < dataList.size(); i += batchSize) {
8
    int start = i;
9
    int end = Math.min(i + batchSize, dataList.size());
10

11
    Future<?> future = executor.submit(() -> {
12
        List<ArticleData> batch = dataList.subList(start, end);
13
        Database db = Database.Static.getDefault();
14

15
        db.beginWrites();
16
        try {
17
            for (ArticleData data : batch) {
18
                Article article = new Article();
19
                // ... populate article
20
                article.save();
21
            }
22
            db.commitWrites();
23
        } finally {
24
            db.endWrites();
25
        }
26
    });
27

28
    futures.add(future);
29
}
30

31
// Wait for all batches to complete
32
for (Future<?> future : futures) {
33
    try {
34
        future.get();
35
    } catch (ExecutionException e) {
36
        System.err.println("Batch failed: " + e.getCause().getMessage());
37
    }
38
}
39

40
executor.shutdown();

Distributed Locks

For unique constraint enforcement across distributed systems:

1
import com.psddev.dari.db.Record;
2

3
public class Article extends Record {
4

5
    @Indexed(unique = true)
6
    private String slug;
7

8
    @Override
9
    protected void beforeSave() {
10
        // DB plugin automatically acquires a distributed lock
11
        // for unique fields during save
12
    }
13
}

Transaction Patterns

Unit of Work Pattern

1
Database db = Database.Static.getDefault();
2

3
db.beginWrites();
4
try {
5
    Article article = Query.from(Article.class)
6
        .where("_id = ?", articleId)
7
        .first();
8

9
    if (article == null) {
10
        throw new IllegalArgumentException("Article not found");
11
    }
12

13
    // Make multiple related changes
14
    article.setPublishDate(new Date(db.now()));
15
    article.setTitle("My Article");
16
    article.save();
17

18
    // Update author stats
19
    Author author = article.getAuthor();
20
    author.incrementPublishedCount();
21
    author.save();
22

23
    // Create activity log
24
    ActivityLog log = new ActivityLog();
25
    log.setType("article_published");
26
    log.setArticle(article);
27
    log.setTimestamp(new Date(db.now()));
28
    log.save();
29

30
    // All changes committed together
31
    db.commitWrites();
32

33
} finally {
34
    db.endWrites();
35
}

Saga Pattern (Compensating Transactions)

1
Database db = Database.Static.getDefault();
2
boolean inventoryReserved = false;
3
boolean paymentProcessed = false;
4

5
try {
6
    db.beginWrites();
7

8
    // Step 1: Reserve inventory
9
    reserveInventory(order);
10
    inventoryReserved = true;
11

12
    // Step 2: Process payment
13
    processPayment(order);
14
    paymentProcessed = true;
15

16
    // Step 3: Create order
17
    order.setStatus("confirmed");
18
    order.save();
19

20
    db.commitWrites();
21

22
} catch (Exception e) {
23
    // Compensating actions
24
    if (paymentProcessed) {
25
        refundPayment(order);
26
    }
27
    if (inventoryReserved) {
28
        releaseInventory(order);
29
    }
30
    throw e;
31

32
} finally {
33
    db.endWrites();
34
}

Optimistic Locking

1
Database db = Database.Static.getDefault();
2

3
db.beginWrites();
4
try {
5
    // Load current version
6
    VersionedArticle current = Query.from(VersionedArticle.class)
7
        .where("_id = ?", article.getId())
8
        .first();
9

10
    if (current.getVersion() != article.getVersion()) {
11
        throw new ConcurrentModificationException(
12
            "Article was modified by another transaction"
13
        );
14
    }
15

16
    // Update with new version
17
    article.setTitle("New Title");
18
    article.setVersion(article.getVersion() + 1);
19
    article.save();
20

21
    db.commitWrites();
22

23
} finally {
24
    db.endWrites();
25
}

Performance Optimization

Batch vs. Individual Commits

1
Database db = Database.Static.getDefault();
2

3
// Slow - many small transactions
4
for (Article article : articles) {
5
    db.beginWrites();
6
    try {
7
        article.save();
8
        db.commitWrites();
9
    } finally {
10
        db.endWrites();
11
    }
12
}
13

14
// Fast - one large transaction
15
db.beginWrites();
16
try {
17
    for (Article article : articles) {
18
        article.save();
19
    }
20
    db.commitWrites();
21
} finally {
22
    db.endWrites();
23
}

Eventual Consistency for Bulk Operations

1
Database db = Database.Static.getDefault();
2

3
// Use eventual consistency for better performance
4
db.beginWrites();
5
try {
6
    for (Article article : articles) {
7
        article.save();
8
    }
9
    db.commitWritesEventually();  // Faster than commitWrites()
10
} finally {
11
    db.endWrites();
12
}

Skip Validation for Trusted Data

1
Database db = Database.Static.getDefault();
2

3
// Import from trusted source - skip validation
4
db.beginWrites();
5
try {
6
    for (ArticleData data : trustedData) {
7
        Article article = convertToArticle(data);
8
        db.saveUnsafely(article.getState());  // No validation
9
    }
10
    db.commitWrites();
11
} finally {
12
    db.endWrites();
13
}

Best Practices

1. Always Use try-finally

1
Database db = Database.Static.getDefault();
2

3
// Good
4
db.beginWrites();
5
try {
6
    article.save();
7
    db.commitWrites();
8
} finally {
9
    db.endWrites();
10
}
11

12
// Bad - endWrites() might not be called
13
db.beginWrites();
14
article.save();
15
db.commitWrites();
16
db.endWrites();

2. Keep Transactions Short

1
Database db = Database.Static.getDefault();
2

3
// Good - short transaction
4
db.beginWrites();
5
try {
6
    article.save();
7
    db.commitWrites();
8
} finally {
9
    db.endWrites();
10
}
11

12
// Bad - long transaction holding locks
13
db.beginWrites();
14
try {
15
    article.save();
16
    sendEmail(article);  // External I/O
17
    updateSearchIndex(article);  // Slow operation
18
    db.commitWrites();
19
} finally {
20
    db.endWrites();
21
}

3. Use Appropriate Commit Strategy

1
Database db = Database.Static.getDefault();
2

3
// Critical data - immediate consistency
4
db.beginWrites();
5
try {
6
    payment.save();
7
    db.commitWrites();  // Must be immediately visible
8
} finally {
9
    db.endWrites();
10
}
11

12
// Non-critical data - eventual consistency
13
db.beginWrites();
14
try {
15
    viewCount.save();
16
    db.commitWritesEventually();  // Performance > consistency
17
} finally {
18
    db.endWrites();
19
}

4. Don't Catch and Ignore Exceptions

1
Database db = Database.Static.getDefault();
2

3
// Bad - hides errors
4
db.beginWrites();
5
try {
6
    article.save();
7
    db.commitWrites();
8
} catch (Exception e) {
9
    // Ignored!
10
} finally {
11
    db.endWrites();
12
}
13

14
// Good - handle or propagate
15
db.beginWrites();
16
try {
17
    article.save();
18
    db.commitWrites();
19
} catch (ValidationException e) {
20
    System.err.println("Validation failed: " + e.getMessage());
21
    throw e;
22
} finally {
23
    db.endWrites();
24
}