Basic Architecture & Goals

In this post I want to explain the basic structure of the project and what my plans are in the near future:

Important (non-self-explanatory parts) from left to right:

Hibernate Search Engine: The base of any Hibernate Search implementation. This contains all the classes necessary to control a Lucene Index the same way as Hibernate Search ORM does.

Hibernate Search Standalone: A standalone implementatation of a SearchIntegrator that can be used to use Hibernate Search's engine without any restrictions about where entities come from.

Hibernate Search Database Utilities: These utilities are used to create the necessary triggers on the database to keep track of Entity changes.

The "main" module - Hibernate Search GenericJPA: This module combines the Database utilities and the standalone Hibernate Search implementation to work together with a JPA provider. It accumulates all the information needed to create the event triggers in the database and sets them up. Then, it also starts a polling service that reads the information about changes in the database and it orders the underlying standalone implementation to update the indexes accordingly.

In order to make the usage of this module as easy as possible, the Hibernate Search GenericJPA module has implementations for nearly all the interfaces in org.hibernate.search.jpa. Users shouldn't have to worry about the implementation details or what version of Hibernate Search they are using. Switching Hibernate Search ORM for its generic counterpart (my
project) should be as easy as switching your JPA provider.

Plans for the project in the near future

1. improve the Update system

As described in my earlier post, currently the user has to create specific Entities that map to the tables that store the update information. 

	@Entity
	@Table(name = "Vendor")
	public class Vendor {
	private Long id;
	private String name;
	private List<Game> games;
	@Id
	@GeneratedValue
	public Long getId() {
	return id;
	}
	public void setId(Long id) {
	this.id = id;
	}
	@Column
	@Field(store = Store.YES, index = Index.YES, analyze = Analyze.NO)
	public String getName() {
	return name;
	}
	public void setName(String name) {
	this.name = name;
	}
	@ManyToOne(targetEntity = Game.class)
	@ContainedIn
	public List<Game> getGames() {
	return games;
	}
	public void setGames(List<Game> games) {
	this.games = games;
	}
	}

view raw Vendor.java hosted with ❤ by GitHub

These serve two purposes:

1. To create the actual table and
2. give the Index-Updating service access to the database via JPA.

This is unnecessary boiler-plate code input that is needed from the user in order to get the whole system to work properly. A new system that serves the same purposes has to be designed, while not forcing the user to provide boiler-plate code.

2. implement more TriggerSources

since Triggers are not part of the SQL specification, every RDBMS handles them a bit different. That's the reason the Database utilities contain a interface called TriggerSQLStringSource that hosts several methods that generate the SQL code needed to create the triggers and set up the database for the Update system. As of now, only a MySQLTriggerSQLStringSource exists. This has to change.

3. easier startup

as of now the user has to start the SearchFactory on his own after he has provided it with the configuration data. While this might be practicable for small projects, this could be a problem for bigger systems in a Enterprise Environment. That's why we have to think of a way to hook into the JPA startup process.

[Current state] How to use the current Version of Hibernate-Search Generic JPA

As promised I am following up with the current state of development in this blog entry.

Firstly, let's take a look at what standard Hibernate Search annotated entities would look like:
First our Game Entity:

	@Indexed
	@Entity
	@Table(name = "Game")
	public class Game {
	private Long id;
	private String title;
	private List<Vendor> vendors;
	public Game() {
	}
	public Game(String title) {
	this.title = title;
	}
	@Id
	@GeneratedValue
	public Long getId() {
	return id;
	}
	public void setId(Long id) {
	this.id = id;
	}
	@Field(store = Store.YES, index = Index.YES, analyze = Analyze.NO)
	@Column
	public String getTitle() {
	return title;
	}
	public void setTitle(String title) {
	this.title = title;
	}
	@IndexedEmbedded(includeEmbeddedObjectId = true, targetElement = Vendor.class)
	@OneToMany
	public List<Vendor> getVendors() {
	return vendors;
	}
	public void setVendors(List<Vendor> vendors) {
	this.vendors = vendors;
	}
	}

view raw Game.java hosted with ❤ by GitHub

And the Vendor Entity:

	@Entity
	@Table(name = "Vendor")
	public class Vendor {
	private Long id;
	private String name;
	private List<Game> games;
	@Id
	@GeneratedValue
	public Long getId() {
	return id;
	}
	public void setId(Long id) {
	this.id = id;
	}
	@Column
	@Field(store = Store.YES, index = Index.YES, analyze = Analyze.NO)
	public String getName() {
	return name;
	}
	public void setName(String name) {
	this.name = name;
	}
	@ManyToOne(targetEntity = Game.class)
	@ContainedIn
	public List<Game> getGames() {
	return games;
	}
	public void setGames(List<Game> games) {
	this.games = games;
	}
	}

view raw Vendor.java hosted with ❤ by GitHub

To get this to work properly in the current state of my Generic JPA version an additional @InIndex is needed at the class-level:

	@InIndex
	@Entity
	@Table(name = "Vendor")
	public class Vendor {
	//...
	}

view raw Additional @InIndex Vendor hosted with ❤ by GitHub

	@InIndex
	@Indexed
	@Entity
	@Table(name = "Game")
	public class Game {
	//...
	}

view raw Additional @InIndex Game hosted with ❤ by GitHub

This is a requirement when working with the general JPA specification as some implementors create subclasses for each Entity and Hibernate-Search needs to know which class in the hierarchy was the original one that was supposed to be in the Index.

At the moment we configure our Hibernate Search instance by subclassing JPASearchFactory. While this might be a bit unconvenient for production use it gets the job done easily during development. Later on this will be possible to be configured via a properties/xml file.

Note that this is an example for a SearchFactory in a Java EE application:

	//make sure this is constructed at Container startup
	@Singleton
	@Startup
	public class EJBSearchFactory extends SQLJPASearchFactory {
	@Resource
	private ManagedScheduledExecutorService exec;
	@PersistenceUnit
	private EntityManagerFactory emf;
	/**
	* since we can not hook into the JPA lifecycle for now
	* we have to make sure the SearchFactory is started.
	* This is done via super.init();
	*/
	@PostConstruct
	public void startup() {
	super.init();
	//register this SearchFactory in Search
	Search.setup( this );
	}
	/**
	* since we can not hook into the JPA lifecycle for now
	* we have to make sure the SearchFactory is stopped as well.
	* This is done via super.shutdown();
	*/
	@PreDestroy
	public void shutdown() {
	super.shutdown();
	}
	@Override
	public void updateEvent(List<UpdateInfo> arg0) {
	//callback, only useful for custom update event processing
	}
	@Override
	protected int getBatchSizeForUpdates() {
	//how many update-information entities should be polled from
	//the database at once
	return 2;
	}
	@Override
	protected String getConfigFile() {
	//Hibernate Search Engine properties are configured in this file
	return "/hsearch.properties";
	}
	@Override
	protected long getDelay() {
	//the delay between update polling
	return 500;
	}
	@Override
	protected TimeUnit getDelayUnit() {
	//the timeunit for the delay
	return TimeUnit.MILLISECONDS;
	}
	@Override
	protected EntityManagerFactory getEmf() {
	//-> the EMF used in this application
	return this.emf;
	}
	@Override
	protected List<Class<?>> getIndexRootTypes() {
	//the Root types
	return Arrays.asList( Game.class );
	}
	@Override
	protected TriggerSQLStringSource getTriggerSQLStringSource() {
	//the trigger source to be used to create the updates triggers
	//in the database (will not be discussed in this blogpost)
	return new MySQLTriggerSQLStringSource();
	}
	@Override
	protected List<Class<?>> getUpdateClasses() {
	//the updates classes (-> later in this post)
	return Arrays.asList( GameUpdates.class, VendorUpdates.class, GameVendorUpdates.class );
	}
	@Override
	protected boolean isUseJTATransaction() {
	return true;
	}
	@Override
	protected ScheduledExecutorService getExecutorServiceForUpdater() {
	//this is needed to have a transaction environemtn for the updater thread
	//for a JTA transaction this must be a ManagedScheduledExecutorService
	return this.exec;
	}
	@Override
	protected Connection getConnectionForSetup(EntityManager em) {
	return em.unwrap(Connection.class);
	}
	}

view raw EJBSearchFactory.java hosted with ❤ by GitHub

The only thing that is needed to get this whole thing to work are the Updates classes. As of now the user has to create these on his own. This will hopefully not be a requirement in the future, anymore.

	@Entity
	@Table(name = "GameUpdates")
	@Updates(tableName = "GameUpdates", originalTableName = "Game")
	public class GameUpdates {
	@Id
	private Long id;
	@IdFor(entityClass = Game.class, columns = "gameId", columnsInOriginal = "id")
	@Column
	private Long gameId;
	@Event(column = "eventType")
	@Column
	private Integer eventType;
	public Long getId() {
	return id;
	}
	public void setId(Long id) {
	this.id = id;
	}
	public Long getGameId() {
	return gameId;
	}
	public void setGameId(Long gameId) {
	this.gameId = gameId;
	}
	public Integer getEventType() {
	return eventType;
	}
	public void setEventType(Integer eventType) {
	this.eventType = eventType;
	}
	}

view raw GameUpdates.java hosted with ❤ by GitHub

	@Entity
	@Table(name = "VendorUpdates")
	@Updates(tableName = "VendorUpdates", originalTableName = "Vendor")
	public class VendorUpdates {
	@Id
	private Long id;
	@IdFor(entityClass = Vendor.class, columns = "vendorId", columnsInOriginal = "id")
	@Column
	private Long vendorId;
	@Event(column = "eventType")
	@Column
	private Integer eventType;
	public Long getId() {
	return id;
	}
	public void setId(Long id) {
	this.id = id;
	}
	public Long getVendorId() {
	return vendorId;
	}
	public void setVendorId(Long vendorId) {
	this.vendorId = vendorId;
	}
	public Integer getEventType() {
	return eventType;
	}
	public void setEventType(Integer eventType) {
	this.eventType = eventType;
	}
	}

view raw VendorUpdates.java hosted with ❤ by GitHub

	@Entity
	@Table(name = "Game_VendorUpdates")
	@Updates(tableName = "Game_VendorUpdates", originalTableName = "Game_Vendor")
	public class GameVendorUpdates {
	@Id
	private Long id;
	@IdFor(entityClass = Game.class, columns = "gameId", columnsInOriginal = "game_ID")
	@Column
	private Long gameId;
	@IdFor(entityClass = Vendor.class, columns = "vendorId", columnsInOriginal = "vendors_ID")
	@Column
	private Long vendorId;
	@Event(column = "eventType")
	@Column
	private Integer eventType;
	public Long getId() {
	return id;
	}
	public void setId(Long id) {
	this.id = id;
	}
	public Long getGameId() {
	return gameId;
	}
	public void setGameId(Long gameId) {
	this.gameId = gameId;
	}
	public Integer getEventType() {
	return eventType;
	}
	public void setEventType(Integer eventType) {
	this.eventType = eventType;
	}
	}

view raw GameVendorUpdates.java hosted with ❤ by GitHub

Note that the user not only has to create a table for each Entity, but also one for each mapping table as well. For an existing database the user will have to index a manually for now (However: All data that is persisted AFTER the setup was done will be updated automatically):

	FullTextEntityManager fem = Search.getFullTextEntityManager( this.em );
	fem.beginSearchTransaction();
	Game newGame = new Game( "Legend of Zelda" );
	fem.index( newGame );
	fem.commitSearchTransaction();

view raw IndexNew.java hosted with ❤ by GitHub

Now that we have set all this up correctly, we can search just like with normal Hibernate Search ORM:

	FullTextEntityManager fem = Search.getFullTextEntityManager( this.em );
	FullTextQuery fullTextQuery = fem.createFullTextQuery( new TermQuery( new Term( "title", "Legend of Zelda" ) ), Game.class );
	List<Game> result = fullTextQuery.getResultList();

view raw QueryExample.java hosted with ❤ by GitHub

This is it for now. Stay tuned for further updates!

Happy Coding,
Martin