low latency Flashcards
how to create immutable objects in java
if you variables are not provided any setters
and
if they are declared as final ( which implies they need to initialized within the constructor or defaulted to a value )
Take care the variable type say Employee has Address .. address should also be immutable
what benefit does creating immutable objects provide
As well they are side-effect free and can be freely passed across multiple threads without them being affected.
why is String object immutable in java
String is statistically the most used data structure in java so it needed optimization
a) String pools are made to reuse String literals so lesser object with same values are created . This helps save memory .. but then that necessitates that they be immutable so that multiple references can point to the same object and be sure its value is not changed
b) anything immutable can be cached safely
c) String also then becomes threadsafe
d) Security .. if another thread calls and changes the value of a string then that doesnt affect you .
void criticalMethod(String userName) {
// perform security checks
if (!isAlphaNumeric(userName)) {
throw new SecurityException();
}
// this security check would be useless if someone can change the value of userNAme after this check is done
// do some secondary tasks
initializeDatabase();
// critical task
connection.executeUpdate("UPDATE Customers SET Status = 'Active' " +
" WHERE UserName = '" + userName + "'"); } 4) Helps in performance whereby we have decreased the size of heap being used. In java 9 they added compaction of string from char[] to be a byte[] in case you are using latin1 characters only
5) Hashcode of String object is only computed once and then its cached. Subsequent call to hashcode returns the cached value .. This in turn helps make collections like hashmap/ hashtable and hashset faster
java 17 new features
pattern matching for switch and instanceof
the case statement doesnt have to be a constant value .. and swtich can work across objects and also initialise the variabel i in this case .
public static String getInfo(Object object) {
return switch (object) {
default -> “Unknown”;
case Integer i -> (i < 0 ? “Negative” : “Positive”)+ “ integer”;
case String s -> “String of length “ + s.length();
case null -> “Null!”;
};
}
whats a race condition
Anytime the behavior of a multi threaded application depends on the timing of the interleaving threads giving inconsistent results can be called a race condition
Especially true for all check then act operation
For example c++
is equal to
read value of c
increment that value by 1
assign new value to C
In case multiple threads are involved and they run you may get an output of 1 ,1 or 1,2 with an input of 0
Lazy initialisation also has this issue
check if object exists and if not create an object
what problem does volatile variables address
has to do with the java memory model. the idea of threads working with their own thread cached shared resources.
Essentially there is a Visibility problem that means other threads don’t see the changes that were made by one thread.
The thread cache and main memory values might differ.
volatile bypasses caching and values are updated in the main memory … and each thread gets the updated value .. they dont have re-read it
low level observers.. each thread using a volatile variable would refresh all values in the current scope when the volatile variables are updated.
Atomic variables also address the visibility issues.( similar to volatile)
Write about Atomic Integers and the non-blocking concurrency setup
AtomicInteger or AtomicReference have added implementation for the compareandswap functionality
Typical use
AtomicInteger i = new AtomicInteger(0)
do{
int oldvalue= i.get() ;
int newvalue =i++;
}
while (!i.compareAndSwap(oldvalue, newvalue)
so eventually the value is set by multiple threads. .CAS returns true or false based on if it was able to do the swap
The AtomicInteger has functions like
getandIncrement or incrementandGet() that work on the above principle and hence can be used in multi threaded application.
-XX:+UseStringDeduplication
G1 collector option only
JVM will try to eliminate duplicate strings as part of garbage collection process
Works only on long lived objects
To see String deduplication statistics, such as how much time it took to run, how much duplicate strings were evacuated, how much savings you gained, you may pass ‘-XX:+PrintStringDeduplicationStatistics’
advantages of keeping xms and xmx the same
system starts up faster
there is no allocation and deallocation of memory from and to the OS as per systems usage of the memory
the max memory is retained for the process at startup so new heavy neighbours wont affect you
heap dump tool
jmap
load heapdum to eclipse MAT or jprofiler/Yourkit
usual cause of memory usages being high
, considerable amount memory is wasted because of inefficient programming practices such as: Duplicate object creation, suboptimal data type definitions (declaring ‘double’ and assigning only ‘float’ values), over allocation and underutilization of data structures and several other practices.
boxed wrappers take a whole lot of memory when compared to primitives along with duplicate strings
you can consider interning strings in your code
you can consider using primitive as much as possible
there are specialised collection framworks from apache that allows primitive values and they are way faster
fastutils is one such
get rid of boxed numbers
GC logging
Enabling GC logging on your application adds almost zero overhead
The GC log rich set of information, such as timestamps of GC events, types of events (Young GC, Full GC, Mixed GC), duration of events, sizes of internal JVM memory regions (Young Gen, Old Gen, Metaspace) before and after the event, time spent in the JVM and Kernel during GC events, number of GC threads used, duration each GC thread took to complete, string deduplication details…
how do you choose a GC
if memory is higher than 32
then
a) if GC throughput is important then go for parallelgc
b) if low pause is a requirement then go for shenandoah or Z GC
if memory is less than 32 then go for
a) if gc throughput isimportant then go for paralllelgc
b) else go for G1 GC.
What do you get by doing GC tuning /
what are the benefits of gc tuning
by reducing the gc pause you can improve response times
can improve throughput
can right size the memory / hence cost
can right size the cpu by reducing the cpu used by GC /hence cost
unearth memory problems earlier by monitoring gc logs
look at GC thread cout in a threaddump
XX:ParallelGCThreads=n: Sets the number of threads used in parallel phase of the garbage collectors.
-XX:ConcGCThreads=n: Controls the number of threads used in concurrent phases of garbage collectors.
So if your JVM is running on server with 32 processors, then
ParallelGCThread value is going to be: 23(i.e. 8 + (32 – 8)*(5/8))
ConcGCThreads value is going to be: 6 (i.e. max(25/4, 1)
how to tune GC
start from scratch .. old jvm arguements added at the first iteration may be obsoleted or counterproductive with newer algorithms or not meet the currect system requirements. There are 600 vm arguments related to memory and gc
. Tune GC Algorithm Settings
Adjust Internal Memory Regions Size
a. Young Generation
b. Old Generation
c. MetaSpace
d. Others
Reduce Object Creation rate : this is at an application level
Tuning G1 GC what do you choose
first and foremost
pause time goal.: ‘-XX:MaxGCPauseMillis’ or something
2)Avoid setting young gen size
3) Remove old arguments
4) XX:+UseStringDeduplication’ argument.
seconds
no of parallelgc threads
no of concurrent gc threads
at what threshold is gc triggered
the size of the region itself
overall heap size
https://blog.gceasy.io/simple-effective-g1-gc-tuning-tips/#:~:text=G1%20GC%20is%20an%20adaptive%20garbage
Memory Analysis toolkit
jps : to find the process ids of java processes in general
jmap -histio : to generate an object histogram of sorts . a snapshot of objects in memory for a PID
jmap is also used to get heapdump
jmap -dump :file=<filelocation> PID</filelocation>
we can also set XX: heapdumponOOM property for the jvm
Eclipse MAT is great for analysiing the hprof file generated by heapdump .. is lightweight
JvisaulVM .. not shipped with jdk 9 onwards can be used to do memory profile / take a dump of threads/ heap etc.
Java mission control is licensed by Oracle and is great for profiling of apps but not available as part of OpenJDK
What is memory leak
memory leak is when an object allocated in memory but no longer needed is not getting garbage collected
In an OOM / heap dump scenario you look at what is Retained Heap size
Shallow heap size is the size of the object itself
Retained is the size of all the objects and their references . So you want to target those with highest Retained heap size
in memory profiling you look at the generational count of objects ..
a count of 0 : object has not gone through any round of gc
1 : one round
if you notice that the generational count is increasing constantly then that can be an indicator of an object not getting garbage collected at all . worth investigation
visualvm also allows to view stack traces to see where in the code that object was created.
most common causes of memory leak
1) Memory Leak Through static Fields
In Java, static fields have a life that usually matches the entire lifetime of the running application
If collections or large objects are declared as static, then they remain in the memory throughout the lifetime of the application.
2) Unclosed resources
connections/input streams / sessions should be closed carefully thinking about scenarios where an exception might lead to these not getting closed properly
close in finally block or use try with resources thing
how improper equals and hashcode can lead to memory leak?
Improper equals() and hashCode() Implementations
Something like this would add 100 objects of the same type. the heap size is 100 instead of 1.
@Test
public void givenMap_whenEqualsAndHashCodeNotOverridden_thenMemoryLeak() {
Map<Person, Integer> map = new HashMap<>();
for(int i=0; i<100; i++) {
map.put(new Person(“jon”), 1);
}
Assert.assertFalse(map.size() == 1);
}
ORM tool like Hibernate, uses the equals() and hashCode() methods to analyze the objects and saves them in the cache. IT may end up saving duplicates in cache
When defining new entities, always override the equals() and hashCode() methods.
It’s not enough to just override, these methods must be overridden in an optimal way as well.
Other memory leaks
. Inner Classes That Reference Outer Classes
non-static inner classes (anonymous classes). keeps an implicit reference to its containing class
If we use this inner class’ object in our application, then even after our containing class’ object goes out of scope, it won’t be garbage collected.
solution : use ZGC or make the inner class static.
Finalize
The finalize() method is called the finalizer. Its defined just as any other method in which you try to close resources and GC has to invoke this method before doing actual GC.
Usually it doesnt help GC that it has to wait and try to execute this first . 8
public void finalize() {
try {
reader.close();
System.out.println(“Closed BufferedReader in the finalizer”);
} catch (IOException e) {
// …
}
}
Threadlocals and memory leak ?
threadlocals are values/references associate with a thread.
In some applications/frameworks that use threadpools.. the threads once created serve multiple request and stay alive for real long .
hence Threadlocal objects can stay alive in memory even though they are no longer being used ..
Solution : Remove the threadlocal value once you have used the threadlocal.get or in the finally threadlocal.remove()
Imagine you had a ThreadLocal<Arraylist<String>> locallist and it stays in memory even though not being relevant anymore!</String>
define latency and throughput
Latency is the time required to perform some action or to produce some result. Latency is measured in units of time – hours, minutes, seconds, nanoseconds or clock periods.
Throughput is the number of such actions executed or results produced per unit of time. This is measured in units of whatever is being produced (cars, motorcycles, I/O samples, memory words, iterations) per unit of time
Generally, you should aim for maximal throughput with acceptable latency.
What is CAP theroem
http://ksat.me/a-plain-english-introduction-to-cap-theorem
In a system which essentially does a read-write operation and is distributed ( so several node separated by network and heterogenous nodes ) . The network in this case is asynchronous that is the nodes can get disconnected( aka partitoned) you always have to choose between consistency and availability
Consistency : When a read is done it will always get the value of the last write . or an Atomic read. If the system cant provide you the data it will respond with an error ( which contradicts Availability)
Availability : The System is always available and you will always get a response . The response may not be the most updated data ( since that data is on another node and due to partition(that node being disconnected from you) has not reached you . You choose to respond in all cases.
C is important
Banking and Financial Systems
Explanation: Banking systems must ensure that all transactions are consistently recorded. For instance, if a bank transaction is processed (e.g., a transfer of funds), it’s critical that the most recent account balance is reflected in all subsequent reads across the system.
Partition Handling: During a network partition, such a system would rather deny access or delay responses than return outdated data, ensuring that the integrity of data is maintained.
Drawback: This approach may impact availability if the system needs to enforce consistency during network partitions, potentially blocking operations until partition issues are resolved
Another example
Inventory Management in E-commerce
Explanation: Real-time inventory management systems need to ensure that products are not oversold. When an item is sold, it’s important that the inventory count is consistent across the entire system to prevent selling more items than available.
Systems Prioritizing Availability (A > C)
When uninterrupted service is critical, even at the cost of possibly returning slightly stale data, systems prioritize Availability over Consistency.
Social media platforms like Twitter, Instagram, or Facebook focus on providing fast, real-time access to updates and notifications, allowing users to continue interacting with the platform even if some data might not reflect the latest update.
Why Availability?: In this context, it’s better for users to have some data available (even if slightly stale) than to block access until the latest updates are fully synchronized.
Partition Handling: During a network partition, each partition may operate independently, allowing users to continue reading and posting content with the expectation that the system will eventually achieve consistency.
Another A
Caching Systems (e.g., CDN for Video Streaming)
Explanation: Content delivery networks (CDNs) prioritize availability, as their primary goal is to serve content quickly to users worldwide. Data is often cached across distributed servers to minimize latency.
Why Availability?: Video streaming platforms like Netflix, YouTube, or Hulu cache popular content at various edge locations to ensure fast access, even if the data is not always the absolute latest (e.g., metadata like the number of views might lag slightly).
How does immutability of objects help and how is that achieved in java
dont provide setters for any field
Make all fields final and private. they are initialized by the constructor or before hand.
Make the class Final so subclassed cant over ride this.
Better way is to make the constructor private and construct instances in factory methods.
If the instance fields include references to mutable objects, don’t allow those objects to be changed:
Don’t provide methods that modify the mutable objects.
Don’t share references to the mutable objects. Never store references to external, mutable objects passed to the constructor; if necessary, create copies, and store references to the copies.
Similarly, create copies of your internal mutable objects when necessary to avoid returning the originals in your methods.
What are benefits of Immutable objects
These objects can be used as Key for Maps and be stored in Sets correctly
If not immutable . the hash value for a mutable object may change and finding them in map or set will fail leading to new objects being added to the collection
Immutable objects are threadsafe because their state never changes and they make for much more predictable code behavior . easier to read and understand the code.
practical use case of immutability
Practical Use Cases for Immutable Objects in Multi-Threading
Shared Configuration Data:
As shown in the example above, immutable objects are ideal for storing application-wide configuration data shared across threads.
Caching:
Immutable objects are often used in caching scenarios where multiple threads read cached data without needing synchronization.
Message Passing:
In producer-consumer systems or message queues, immutable objects ensure that messages passed between threads cannot be modified.
Functional Programming:
Immutable objects are foundational in functional programming paradigms (e.g., Java Streams API), where operations like mapping and filtering do not modify the original data but create new instances.
Top 5 things to do for low latency
Garbage Collection (GC): The introduction of ZGC in Java 11 and G1GC enhancements in Java 17 have made GC pauses extremely low and consistent, which is a key factor for low latency applications. ZGC is designed specifically for low latency, scalability, and ease of use, allowing a Java application to continue running while it performs all GC operations except thread stack scanning.
Immutable Objects: Making objects immutable can help reduce the amount of synchronization required and can improve performance, especially in highly concurrent applications.
Concurrency Utilities: Java 5 introduced several concurrency utilities and locks that can help improve performance in multi-threaded applications. Examples include the Lock interface, CountDownLatch, and Semaphore classes.
Profiling: Using a profiler can help identify performance bottlenecks and optimize code for low latency. Tools like JProfiler, YourKit, and VisualVM can provide insights into CPU usage, memory usage, and thread behavior.
Amdahl’s Law: Amdahl’s Law can be used to identify the sequential execution path in a program and optimize it for performance. By improving the performance of the critical path, overall performance can be improved.
Warmup: JVM warmup can be used to improve the performance of Java applications by allowing the JIT compiler to optimize code before it is executed. This can reduce the time required for JIT compilation during runtime and improve overall performance.
Z Garbage Collector (ZGC)
is a scalable low latency garbage collector
ZGC performs all expensive work concurrently, without stopping the execution of application threads for more than a few milliseconds .. under milleseconds in some cases
Pause times are independent of heap size that is being used
works well for most heap size ( 8 MB to 16 TB) .
The most important tuning option for ZGC is setting the max heap size (-Xmx)
ZGC is a concurrent collector
think live set ( currently active objects ) and the object allocation rate ( new objects being created)
The second tuning option one might want to look at is setting the number of concurrent GC threads (-XX:ConcGCThreads)
ZGC has heuristics to automatically select this number
ZGC uncommits unused memory, returning it to the operating system. This is useful for applications and environments where memory footprint is a concern
Z1GG
Garbage 1st collector
It divides the entire heap into regions . say of 1 MB each .
Any region maybe used for eden/ survivor or old gen
The marking of live objects happens concurrently with the running application (there are slight pauses when they start )
The evacuation or collection of regions is done by other threads which run parallel with the application ( they do cause a pause)
The evacuation will pick regions with the highest garbage and move the live objects out to another region ( young -> survivor ) or survivor o old or old to old
The region that got cleaned up is now made available for any assignment
The tuning is in terms of specifying the maxGCpausetime in milliseconds
The GC is adaptive to see how many regions it can clean up while staying within the gc pause time limit. Its not always guaranteed but it tries
Z1GC does provide balanced low latency ( 200 ms to seconds) and high throughput.. Its quite balanced.
Tuning can be done to the number of concurrent threads doing marking
and parallel threads doing evacuation
It has a young phase collection ( young to old) .. more tuned to faster runtime
Mixed Collection : Old and new .. more tuned to clearing up maximum memory. does have small pause.
There is also the full GC that can happen when there isnt space to allocate new objects .. like an allocation failure
Reading G1GC logs
[Eden: 3072.0K(194.0M)->0.0B(201.0M) Survivors: 0.0B->0.0B Heap: 3727.1M(4022.0M)->3612.0M(4022.0M)], [Metaspace: 2776K->2776K(1056768K
2015-09-14T12:32:24.398-0700: 0.356 – Here 2015-09-14T12:32:24.398-0700indicates the time at which this GC event fired. Here 0.356 indicates that 356 milliseconds after the Java process was started this GC event was fired.
GC pause (G1 Evacuation Pause) — Evacuation Pause is a phase where live objects are copied from one region (young or young + old) to another region.
(young) – indicates that this is a Young GC event.
GC Workers: 8 – indicates the number of GC worker threads.
Times: user=0.08, sys=0.00, real=0.02 secs – here note the ‘real’ time, it indicates this GC event took 0.02 seconds to complete. If you are wondering what is the purpose of ‘user’ and ‘sys’ times, please refer to this article.
Using a valueof static factory method
You can cache Objects and return existing objects rather than creating one each time.
For example all a-b and A-Z Character are already cached .
public class CharacterCacheExample {
public static void main(String[] args) {
Character c1 = Character.valueOf(‘A’); // Cached
Character c2 = Character.valueOf(‘A’); // Same cached instance
Same for Boolean
Boolean.TRUE and Boolean.FALSE are predefined static instances of Boolean objects.
If b is true, it returns Boolean.TRUE (a cached instance).
If b is false, it returns Boolean.FALSE (a cached instance
Builder pattern for object creation
Builder pattern is a good choice when designing classes
whose constructors or static factories would have more than a handful of
parameters, especially if many of the parameters are optional or of identical type.
Client code is much easier to read and write with builders than with telescoping
constructors, and builders are much safer than JavaBeans.
Correct Way to Handle File Closing
✅ Use try-with-resources (AutoCloseable)
✅ Explicitly close resources in a finally block if try-with-resources is not an option
Operating systems limit the number of open file descriptors per process (e.g., 1024 on many Linux systems).
If too many files are left open (due to delayed GC cleanup), attempts to open new files will fail with errors like:
Open files consume memory and system resources.
Not closing files immediately can lead to memory leaks and degraded performance.
