在企业开发中可能会遇到这样一种情况,服务器报out of memory异常了,堆Dump文件却看不出问题,这个时候可考虑直接内存溢出问题,也是很容易被忽略的一个点!
《Java虚拟机规范》并没有定义直接内存这块区域,且它也不属于语虚拟机运行时数据区
why直接内存?
使用场景:
Unsafe.allocateMemory(size);
ByteBuffer.allocateDirect(size);
Unsafe可用来直接访问系统内存资源并自主管理,在提升Java运行效率、增强Java语言底层操作能力方面起了很大的作用——可以认为,Unsafe类是Java中留下的后门,提供了一些低层次操作,如直接内存访问、线程调度等。
Unsafe不属于Java标准。官方并不建议使用Unsafe,并且从JDK 9开始,官方开始去除Unsafe(issue)。
因此,Unsafe类对于项目实战,意义并不大。然而目前业界有很多好用的类库大量使用了Unsafe类,例如java.util.concurrent.atomic包里的一堆类、Netty、Hadoop,Kafka等。所以了解一下还是有好处的。
不同的JDK版本中,Unsafe类也有区别:
package com.wjw.jvminaction;
import sun.misc.Unsafe;
import java.lang.reflect.Field;
public class DirectMemoryTest1 {
private static final int MB_1 = 1024 * 1024;
public static void main(String[] args) throws IllegalAccessException {
// 通过反射获取Unsafe类并通过其分配直接内存
Field unsafeField = Unsafe.class.getDeclaredFields()[0];
unsafeField.setAccessible(true);
Unsafe unsafe = (Unsafe) unsafeField.get(null);
// 分配1M内存,并返回这块内存的起始地址
Long address = unsafe.allocateMemory(MB_1);
// 向内存地址中设置对象
unsafe.putByte(address, (byte) 1);
// 从内存中获取对象
byte aByte = unsafe.getByte(address);
System.out.println(aByte);
// 释放内存
unsafe.freeMemory(address);
}
}
https://www.jb51.net/article/140726.htm
https://blog.csdn.net/ahilll/article/details/81628215
https://www.jianshu.com/p/dd2be4d3b88e
package com.wjw.jvminaction;
import sun.misc.Unsafe;
import java.lang.reflect.Field;
public class DirectMemoryTest3 {
private static final int GB_1 = 1024 * 1024 * 1024;
public static void main(String[] args) throws IllegalAccessException {
// 通过反射获取Unsafe类并通过其分配直接内存
Field unsafeField = Unsafe.class.getDeclaredFields()[0];
unsafeField.setAccessible(true);
Unsafe unsafe = (Unsafe) unsafeField.get(null);
int i = 0;
while (true){
unsafe.allocateMemory(GB_1);
System.out.println(++ i);
}
}
}
这里也会报OOM的错,但是没有任何的说明信息
使用如下JVM参数启动:
-XX:MaxDirectMemorySize=100m
发现仍然分配了多次,说明MaxDirectMemorySize无法控制Unsafe类
https://blog.csdn.net/z69183787/article/details/77102198/
package com.wjw.jvminaction;
import java.nio.ByteBuffer;
public class DirectMemoryTest2 {
private static final int ONE_MB = 1024 * 1024;
public static void main(String[] args) {
ByteBuffer buffer = ByteBuffer.allocateDirect(ONE_MB);
// 相对写,向position的位置写入一个byte,并将postion+1,为下次读写做准备
buffer.put("abcde".getBytes());
buffer.put("fghij".getBytes());
// 转换为读取模式
buffer.flip();
// 相对读,从position位置读取一个byte,并将position+1,为下次读写作准备
// 读取第1个字节(a)
System.out.println((char) buffer.get());
// 读取第2个字节(b)
System.out.println((char) buffer.get());
// 绝对读,读取byteBuffer底层的bytes中下标为index的byte,不改变position
// 读取第3个字节(c)
System.out.println((char) buffer.get(2));
}
}
package com.wjw.jvminaction;
import sun.misc.Unsafe;
import java.lang.reflect.Field;
import java.nio.ByteBuffer;
/**
* Created by Enzo Cotter on 2021/8/21.
*/
public class DirectMemoryTest4 {
private static final int GB_1 = 1024 * 1024 * 1024;
public static void main(String[] args) throws IllegalAccessException {
int i = 0;
while (true){
ByteBuffer buffer = ByteBuffer.allocateDirect(GB_1);
System.out.println(++ i);
}
}
}
使用如下JVM参数启动:
-XX:MaxDirectMemorySize=100m
发现上来就报错,证明MaxDirectMemorySize可以控制ByteBuffer类,并且报错有提示信息
进入到 java.nio.ByteBuffer.allocateDirect 方法
public static ByteBuffer allocateDirect(int capacity) {
return new DirectByteBuffer(capacity);
}
DirectByteBuffer(int cap) { // package-private
super(-1, 0, cap, cap);
boolean pa = VM.isDirectMemoryPageAligned();
int ps = Bits.pageSize();
long size = Math.max(1L, (long)cap + (pa ? ps : 0));
Bits.reserveMemory(size, cap);
long base = 0;
try {
base = unsafe.allocateMemory(size);
} catch (OutOfMemoryError x) {
Bits.unreserveMemory(size, cap);
throw x;
}
unsafe.setMemory(base, size, (byte) 0);
if (pa && (base % ps != 0)) {
// Round up to page boundary
address = base + ps - (base & (ps - 1));
} else {
address = base;
}
cleaner = Cleaner.create(this, new Deallocator(base, size, cap));
att = null;
}
异常是 Bits.reserveMemory(size, cap); 产生的
static void reserveMemory(long size, int cap) {
if (!memoryLimitSet && VM.isBooted()) {
maxMemory = VM.maxDirectMemory();
memoryLimitSet = true;
}
// optimist!
if (tryReserveMemory(size, cap)) {
return;
}
final JavaLangRefAccess jlra = SharedSecrets.getJavaLangRefAccess();
// retry while helping enqueue pending Reference objects
// which includes executing pending Cleaner(s) which includes
// Cleaner(s) that free direct buffer memory
while (jlra.tryHandlePendingReference()) {
if (tryReserveMemory(size, cap)) {
return;
}
}
// trigger VM's Reference processing
System.gc();
// a retry loop with exponential back-off delays
// (this gives VM some time to do it's job)
boolean interrupted = false;
try {
long sleepTime = 1;
int sleeps = 0;
while (true) {
if (tryReserveMemory(size, cap)) {
return;
}
if (sleeps >= MAX_SLEEPS) {
break;
}
if (!jlra.tryHandlePendingReference()) {
try {
Thread.sleep(sleepTime);
sleepTime <<= 1;
sleeps++;
} catch (InterruptedException e) {
interrupted = true;
}
}
}
// no luck
throw new OutOfMemoryError("Direct buffer memory");
} finally {
if (interrupted) {
// don't swallow interrupts
Thread.currentThread().interrupt();
}
}
}
它大致的作用是做了一个内存上的校验,看是否满足MaxDirectMemorySize的限制,不满足的话就抛异常。
经过校验后DirectByteBuffer构造函数在 base = unsafe.allocateMemory(size); 这里会真正的分配内存,其实也是用unsafe实现的。
联系客服