When you are building backend REST APIs and if you plan to generate identifiers or primary keys for your table records, you have to pick a primary key generation strategy which could be either one of the following AUTO, IDENTITY, SEQUENCE, TABLE or UUID. When designing a system from scratch, you always would want to pick the strategy that works best for you. I was working on a set of backend REST APIs which are built using Kotlin Spring Hibernate framework with MySQL database. For persistence and ORM I decided to use Spring data JPA for this project, as it is suited the best. Here in this article I wish to share my learnings on use of GenerationType.TABLE vs. GenerationType.SEQUENCE, and determine which id generation strategy is faster when we have to dynamically generate identifiers for table records.

Problem Statement: How to generate String identifiers with a prefix with Spring boot using JPA

As mentioned, I am using MySQL database with Hibernate and Spring data JPA. For few of my tables I had a requirement to have simple numerical identifiers like 1,2,3,4 and so on. But for some of the tables in the database I wished to have an id with prefix like:

I read through a lot of posts available on the internet about how to generate an identifier with prefix and which GenerationType strategy is best for it. In most of the posts it was suggested that GenereationType.IDENTITY or GenerationType.SEQUENCE are fast and scalable enough to give a good performance. But since I wished to generate an ID which is supposed to have string characters as well as integers, IDENTITY type generation strategy was ruled out. As to utilize the true power of GenereationType.IDENTITY, I would need to have an AUTO_INCREMENT column in MySQL DB, which would have defeated the whole purpose of having a string prefix, and columns with VARCHAR cannot have AUTO_INCREMENT property.

Implementing Custom SequenceGenerator with Spring boot JPA using MySQL DB

In some of the posts on the internet it is suggested that identifier generation strategy GenerationType.SEQUENCE is one of the best strategies available. But given our use case we are using MySQL DB in this implementation, and as we know MySQL DB does not support sequences. Therefore if we wish to use SequenceGenerator strategy in Spring boot JPA using MySQL DB, then we need go with some workarounds. Since MySQL does not support the sequences, but still if we want to use this strategy, we have to create a sequence table which would maintain the next_val counter for the records in our table. JPA in Hibernate is built in such a way that SequenceGenerator strategy can work with a table as well in the case of MySQL. So to implement the same one can create a TABLE_NAME_seq table and use it with Spring data JPA in Hibernate. I created following table via SQL:

CREATE TABLE IF NOT EXISTS `cosmos`.`my_table_seq` (
  `name` VARCHAR(45) NOT NULL,
  `next_val` BIGINT(20) NULL DEFAULT NULL,
  PRIMARY KEY (`name`))

Next we need to add a record to this table, to get things started:

Now we need to build a custom Sequence Generator using the SequenceStyleGenerator class:

package com.truiton.cosmos.sequences

import org.hibernate.HibernateException
import org.hibernate.MappingException
import org.hibernate.engine.spi.SharedSessionContractImplementor
import org.hibernate.id.enhanced.SequenceStyleGenerator
import org.hibernate.internal.util.config.ConfigurationHelper
import org.hibernate.service.ServiceRegistry
import org.hibernate.type.Type
import org.hibernate.type.spi.TypeConfiguration
import java.io.Serializable
import java.util.*


class StringPrefixedSequenceIdGenerator : SequenceStyleGenerator() {
    val VALUE_PREFIX_PARAMETER: String = "valuePrefix"
    val VALUE_PREFIX_DEFAULT: String = ""
    private var valuePrefix: String? = null

    val NUMBER_FORMAT_PARAMETER: String = "numberFormat"
    val NUMBER_FORMAT_DEFAULT: String = "%d"
    private var numberFormat: String? = null

    @Throws(HibernateException::class)
    override fun generate(
        session: SharedSessionContractImplementor?,
        `object`: Any?
    ): Serializable {
        return valuePrefix + String.format(numberFormat!!, super.generate(session, `object`))
    }

    @Throws(MappingException::class)
    override fun configure(type: Type?, params: Properties?, serviceRegistry: ServiceRegistry?) {
        super.configure(
            TypeConfiguration().basicTypeRegistry.getRegisteredType(Long::class.java),
            params,
            serviceRegistry
        )
        valuePrefix = ConfigurationHelper.getString(
            VALUE_PREFIX_PARAMETER,
            params, VALUE_PREFIX_DEFAULT
        )
        numberFormat = ConfigurationHelper.getString(
            NUMBER_FORMAT_PARAMETER,
            params, NUMBER_FORMAT_DEFAULT
        )
    }
}

To use this StringPrefixedSequenceIdGenerator, we need reference it in our entity class by using the following:

@Id
@GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "cosmos_sequence1")
@GenericGenerator(
    name = "cosmos_sequence1",
    type = StringPrefixedSequenceIdGenerator::class,
    parameters = [
        org.hibernate.annotations.Parameter(name = "sequence_name", value = "my_table_seq"),
        org.hibernate.annotations.Parameter(name = "increment_size", value = "50"),
        org.hibernate.annotations.Parameter(name = "optimizer", value = "pooled-lo"),
        org.hibernate.annotations.Parameter(name = "valuePrefix", value = "TPA"),
        org.hibernate.annotations.Parameter(name = "numberFormat", value = "%05d")])
@Column(name = "id", nullable = false, length = 10)
open var id: String? = null

Now when we run this, and insert any record in the database for this entity, it will generate the desired identifier format i.e. TPA00001. Also it will maintain the counter for increment in my_table_seq table, while having the pre allocated numbers in the memory. Like for this example it would hit the my_table_seq table once to increment after 50 inserts have happened.

This identifier generation strategy for Spring JPA works well with MySQL. But with this approach the only downside I feel is that, we have to create a sequence table corresponding to every table in our database where we need to maintain sequence strategy based IDs. If only, we could optimize this further, our database would be much cleaner. The answer to this problem is GenerationType.TABLE, which we’ll explore in next section.

Implementing Custom TableGenerator with Spring boot JPA using MySQL DB

Here I will show how to implement a custom TableGenerator which adds a prefix to the generated incremental numeric value in Spring boot JPA using MySQL database. Also I will explain how efficiently this identifier generation strategy can be used for multiple tables when generating primary keys for records.

Internally this GenerationType.TABLE strategy is very similar to sequence generator. Here a single table can have multiple next_val counters for multiple entity tables. There are many threads available over the internet about its performance, but lets reserve this point for next section where we compare GenerationType Table vs Sequence. For now lets create a table where we can maintain primary keys for our entities:

CREATE TABLE IF NOT EXISTS `cosmos`.`cosmos_sequence` (
  `sequence_name` VARCHAR(255) NOT NULL,
  `next_val` BIGINT(20) NULL DEFAULT NULL,
  PRIMARY KEY (`sequence_name`))

Next lets create a class for implementing a custom TableGenerator which can generate primary keys with a prefix in Spring boot JPA with MySQL.

package com.truiton.cosmos.sequences

import org.hibernate.HibernateException
import org.hibernate.MappingException
import org.hibernate.engine.spi.SharedSessionContractImplementor
import org.hibernate.id.enhanced.TableGenerator
import org.hibernate.internal.util.config.ConfigurationHelper
import org.hibernate.service.ServiceRegistry
import org.hibernate.type.Type
import org.hibernate.type.spi.TypeConfiguration
import java.io.Serializable
import java.util.*


class StringPrefixedTableGenerator : TableGenerator() {
    val VALUE_PREFIX_PARAMETER: String = "valuePrefix"
    val VALUE_PREFIX_DEFAULT: String = ""
    private var valuePrefix: String? = null

    val NUMBER_FORMAT_PARAMETER: String = "numberFormat"
    val NUMBER_FORMAT_DEFAULT: String = "%d"
    private var numberFormat: String? = null

    @Throws(HibernateException::class)
    override fun generate(
        session: SharedSessionContractImplementor?,
        `object`: Any?
    ): Serializable {
        return valuePrefix + String.format(numberFormat!!, super.generate(session, `object`))
    }

    @Throws(MappingException::class)
    override fun configure(type: Type?, params: Properties?, serviceRegistry: ServiceRegistry?) {
        super.configure(
            TypeConfiguration().basicTypeRegistry.getRegisteredType(Long::class.java),
            params,
            serviceRegistry
        )
        valuePrefix = ConfigurationHelper.getString(
            VALUE_PREFIX_PARAMETER,
            params, VALUE_PREFIX_DEFAULT
        )
        numberFormat = ConfigurationHelper.getString(
            NUMBER_FORMAT_PARAMETER,
            params, NUMBER_FORMAT_DEFAULT
        )
    }
}

Now to actually use this StringPrefixedTableGenerator we need to reference it in our entity class with all the parameters

@Id
@GenericGenerator(
    name = "my_table_sequence",
    type = StringPrefixedTableGenerator::class,
    parameters = [
        org.hibernate.annotations.Parameter(name = TableGenerator.TABLE_PARAM, value = "cosmos_sequence"),
        org.hibernate.annotations.Parameter(name = TableGenerator.SEGMENT_COLUMN_PARAM, value = "sequence_name"),
        org.hibernate.annotations.Parameter(name = TableGenerator.VALUE_COLUMN_PARAM, value = "next_val"),
        org.hibernate.annotations.Parameter(name = TableGenerator.SEGMENT_VALUE_PARAM, value = "my_table_sequence"),
        org.hibernate.annotations.Parameter(name = "initial_value", value = "1"),
        org.hibernate.annotations.Parameter(name = "increment_size", value = "50"),
        org.hibernate.annotations.Parameter(name = "optimizer", value = "pooled-lo"),
        org.hibernate.annotations.Parameter(name = "valuePrefix", value = "TPA"),
        org.hibernate.annotations.Parameter(name = "numberFormat", value = "%05d")]
)
@GeneratedValue(strategy = GenerationType.TABLE, generator = "my_table_sequence")
@Column(name = "id", nullable = false, length = 10)
open var id: String? = null

When an insert happens through the above code, a record is created in the underlying table which we have specified for the TableGenerator(in this case cosmos_sequence). Once a record is created in cosmos_sequence Spring JPA can perform 50 inserts without referencing this underlying table. This is done by using the pooled-lo optimizer, you can read more about it in its official documentation. The above code would give us the desired identifier format TPA00001.

Just to share this is how your underlying sequence table would look when you would have multiple sequences for multiple entities in the same TableGenerator table:

Therefore, as you can see, this approach would help us in optimizing the number of tables in our database. As we have used only one table for having primary key counters for multiple entities. Also, if you do not wish to have a single table due to fault tolerance or any other factors on specific entities, you can opt for an architecture where groups of specific entities have separate sequence tables while others are combined in to groups of 8, 16 or as per your need and performance.

In many online posts I have read people mentioning about the performance for TableGenerator in comparison to SequenceGenerator, I believe things are a little different when we are building with MySQL database. In the next section I would discuss about the performance of GenerationType.TABLE and GenerationType.SEQUENCE in Hibernate using MySQL database.

Comparing Custom TableGenerator vs. SequenceGenerator performance in Spring boot JPA using MySQL DB

When we compare GenerationType.TABLE vs. GenerationType.SEQUENCE using MySQL DB, there is not much of a difference in the implementation. Firstly, be it SEQUENCE generation strategy or the TABLE generation, both strategies when used with pooled-lo optimization, the primary key is generated in the memory itself after the insert of first record. To be clear, in both the implementations next_val column’s values from the underlying table(s) are fetched only once, either on first insert or after the in-memory values are completely exhausted.

Secondly, both the approaches being discussed here use row level read locking techniques to safeguard the transaction and neither of them apply a table level lock when using MySQL DB with InnoDB. To query the next_val from sequence tables, Spring data JPA uses the primary key in the where clause of the query in case of GenerationType.TABLE, therefore a lock is applied only on that row. While in generation type SEQUENCE strategy only a single record is supposed to be present in the sequence table, therefore only that is locked and it should not make a difference. To actually validate this hypothesis, while the first insert was happening, I logged the queries for both of these primary key generation strategies :

Query for generation type SEQUENCE strategy using MySQL database:

select next_val as id_val from my_table_seq for update
update my_table_seq set next_val= ? where next_val=?
insert into exp_pav_loc (created_on,exp_pav_id,is_deleted,loc_type,name,number,updated_on,id) values (?,?,?,?,?,?,?,?)

Query for GenerationType.TABLE strategy using MySQL database:

select tbl.next_val from cosmos_sequence tbl where tbl.sequence_name=? for update
update cosmos_sequence set next_val=?  where next_val=? and sequence_name=?
insert into exp_pav_loc (created_on,exp_pav_id,is_deleted,loc_type,name,number,updated_on,id) values (?,?,?,?,?,?,?,?,?,?)

As you can see in the above SQL statements both of the primary key generation strategies use SELECT FOR UPDATE statements which locks rows only under normal circumstances. This is the reason, I feel that there is not much of a difference between TableGenerator and SequenceGenerator strategy when using MySQL DB with Spring data JPA. As both of them rely on the same working strategy for maintaining the counters for next_val.

Performance of TableGenerator and SequenceGenerator in Spring boot hibernate using MySQL

To further share data on the claim that there is not much of a difference in performance of GenerationType.TABLE vs. GenerationType.SEQUENCE when using MySQL DB, I created APIs and used JMeter for performance testing. For the first test I created one REST POST API, which performs an insert on a table when called. Further I used JMeter with following load to test its performance

For the first three runs I had configured the API to use GenerationType.SEQUENCE as the primary key generation strategy using the code shared in above sections. I will log the time taken in milliseconds for an API to return response by using JMeter. Please have a look at the 15k sample results for this performance tests:

For next three runs I configured the same API to use GenerationType.TABLE, have a look at the results:

As you can see that results are very similar but for better representation, lets have a look at the average of key figures to compare the performance between SequenceGenerator and TableGenerator

As we can see these results show a very minor difference in milliseconds even at the 99th percentile. But to be completely sure, next I decided to execute a performance test between the GenerationType.SEQUENCE vs. GenerationType.TABLE with three APIs using MySQL DB on Spring data JPA. This will help us understand the behavior of TableGenerator when multiple reads/inserts/updates happen in the same sequence table.

To do this I created three different REST POST APIs, which run an insert statement on the desired table based on the code shared in above sections. Then I configured JMeter to call these APIs sequentially in a thread group, but since I am planning to have multiple threads, eventually all of these would be called in parallel.

For the first three runs I configured the APIs to use GenerationType.SEQUENCE as the primary key generation strategy. Through this performance test I will log their response time in milliseconds. Please have a look at the data gathered, in total I captured roughly 45K samples in 3 runs of 15K each, where internally 3 APIs were being called:

Next, I configured the same set of 3 APIs to use the GenerationType.TABLE strategy to generate primary keys using Spring data JPA in MySQL DB. Like all the tests performed earlier, here as well I will log response time in milliseconds. For this test I captured around 45K samples, please have a look:

As you can see the results are very similar when we compare both the tests for 3 APIs between the SEQUENCE generation strategy and TABLE generation strategy. For better representation, lets have a look at the comparison graph:

Interestingly, even when we have this large range of samples the difference between performance of SequenceGenerator and TableGenerator is very minimal. This test not just only highlights the difference but also certifies that internally the working of SequenceGenerator and TableGenerator is very similar when used for MySQL DB in Spring data JPA.

Difference between GenerationType.SEQUENCE and GenerationType.TABLE

Looking at the data captured in performance tests above, I feel the only major functional difference between GenerationType.SEQUENCE and GenerationType.TABLE when used for MySQL DB via Spring data JPA is that the first one maintains the primary key next_val counter in separate tables vs. the later one stores the same for multiple entities in a single table. Data from the above performance tests also imply that there is no significant difference in performance when either of them is used on MySQL DB with Spring data JPA.

Another advantage of GenerationType.TABLE strategy is that it helps in keeping the database clean, as a single sequence counter table can be used to store the primary key counters for multiple entities. On the contrary, if one plans to use GenerationType.SEQUENCE, they would have to maintain equal number of sequence tables. Like, for example if you have 100 tables where primary keys need to be generated, then you need to have 100 sequence tables as well when using Spring data JPA with GenerationType.SEQUENCE, in MySQL DB.

On the other hand when using TableGenerator, you can chose to optimize according to your need, where you can plan to have one or more than one table for maintaining primary key counters, you can always have groups of entities using same sequence table based on some custom logic. Or if you have a small project you can have single sequence table as well under TableGenerator strategy for all your entities. When using TableGenerator, based on your requirement you can take a decision and build your system. I understand that there are many factors which are considered when a database is chosen for a system, but if you plan to choose MySQL database hope this article helps you decide your ID generation strategy. If this article helped you, please feel free to share it with your friends.

Mohit Gupt

Born in New Delhi, India. A software engineer by profession, an android enthusiast and mobile development evangelist. My motive here is to create a group of skilled engineers, who can build better software. Reason being programming is my passion, and also it feels good to make a device do something you want. Professionally I have worked with many software engineering and product development firms. As of now too, I am employed as a senior engineer in a leading tech company. In total I may have worked on more than 20 projects professionally, but whenever I get spare time I share my thoughts here at Truiton.

www.truiton.com

The post Spring Boot Hibernate – JPA GenerationType Table vs Sequence strategy with MySQL appeared first on Truiton.