Kafka clients are composed of producer and consumer. Understanding consumer is very important for overall architecture. A Topic is composed of several partitions (the number is defined when creating the Topic). And Kafka consumer is composed of group and client. (There could be multi groups and multi clients in a group) Each group and client … Continue reading How Kafka consumer works?
Building Kafka cluster is crucial for the production system. Kafka cluster gives the following advantages. Support for failover in case of a node down Queue replication Support for consumer scale out (standalone Kafka also supports consumer scale out) Step 1 - Build Zookeeper ensemble Kafka depends on Zookeeper for it's configuration management. Therefore, Zookeeper needs … Continue reading Building Kafka cluster
Zookeeper is a cluster management solution. Therefore, Zookeeper needs to be set up as a cluster. If not, it can be the Single Point of Failure. Zookeeper cluster is called "ensemble". How to set up ensemble (based on v3.4.10) Basic config for a single Zookeeper is as follows. To change it into ensemble, add some … Continue reading Building Zookeeper cluster
Tomcat internal cluster session manager can be used on a test system, but it has performance issue on a production system. (The problem of Tomcat session cluster) From my benchmark test, total TPS saturated from 3 instances. (It depends on H/W spec and business logic) Therefore I needed to make a custom session manager which … Continue reading Tips on customizing session manager
Tomcat BackupManager is another cluster session manager. It is also sibling class with DeltaManager. Some important features of BackupManager It is activated by declaring <Manager className="org.apache.catalina.ha.session.BackupManager"> inside <Cluster> tag When an instance create a sesion, that instance becomes the session's primary node. Another node becomes backup node and all the other node become proxy node … Continue reading About Tomcat BackupManager
Some people said that Tomcat session cluster is not fit for high traffic web services. So I did performance test. (both DeltaManager and BackupManager) An ideal cluster is whose total TPS shows linear graph. But Tomcat session cluster showed log graph. (it's coefficient is dependent on usage pattern) I analyzed why it happened. The following … Continue reading The problem of Tomcat session cluster
Servlet session on Tomcat is managed by "Tomcat manager". There are 3 kinds of session manager. Default manager (StandardManager) Delta manager Backup manager Default manager (StandardManager) is non clustered session manager. That is, sessions are not shared between Tomcat instances. Without specific setting, default manager is used when we use servlet session api. Delta manager … Continue reading About Tomcat session cluster