2020年5月

Linux /dev/ 目录下的一些特殊文件目录

总结一些 /dev 目录下的特殊文件或目录

  1. /dev/null
    空文件, 可以往里面丢任何不想要的东西, 从里面读只能对到一个空
  2. /dev/zero
    空文件, 可以往里面丢任何不想要的东西, 从里面读能返回一个一直有空值的流, 一直返回空值
  3. /dev/shm
    shm = shared memory, 是个目录, 在内存里面开辟的tmpfs文件系统, 所以操作极快. 往里面填文件, 很快占满内存.
  4. /dev/random /dev/urandom /dev/arandom
    通过搜集系统驱动和其它源的信息, 产生伪随机数. 上述三个有区别, 比如在是不是 block 操作;

JDK 8 new features

  1. Lambda Expressions

    1. A lambda expression is an anonymous function. A function that doesn’t have a name and doesn’t belong to any class;
    2. Syntax: (parameter_list) -> {function_body} 如果只有1个参数,可以省去括号, 如果函数体只有一句,可以省略大括号;
    3. 一个方法method(函数function)包含4部分: 方法名, 参数列表, 方法体, 返回值类型. 一个 lambda 表达式: 1)没有名字; 2) 有参数列表; 3) 有函数体; 4) 没有返回值类型, 但是函数可以返回值, JVM 动态侦测类型;
    4. Java 8 之前, 很多地方我们使用匿名内部类, 8及之后, 都可以使用 lambda 表达式来替换;
    5. 通常使用 lambda 表达式的地方 -> 函数式接口(只有一个抽象的方法), 比如 Callable, Runnable, 各种 Listener;
  2. Method References 方法引用
    它通常是 lambda 表达式的一种缩写形式 str -> System.out.println(str) -> System.out::println
    4种类型的方法引用:

    1. Method reference to an instance method of an object – object::instanceMethod
    2. Method reference to a static method of a class – Class::staticMethod
    3. Method reference to an instance method of an arbitrary object of a particular type – Class::instanceMethod
    4. Method reference to a constructor – Class::new
  3. Functional Interfaces 函数式接口

    1. An interface with only single abstract method is called functional interface(Single Abstract Method interfaces);
    2. JDK 预先定义的函数式接口 链接;
    3. 可选的 Annotation: @FunctionalInterface 编译时能侦测是不是符合函数式接口定义;
    4. 定义的时候, 除了这个唯一的抽象的方法, 可以有任何多个非抽象的实例方法或静态方法;
    5. Java 8 之前通常使用匿名内部类来实现这种接口, Java 8 及以后可以使用 lambda 表达式;
  4. 接口增加 default method and static method

    1. Java 8 之前接口只能有抽象方法, 所有的方法默认都是 public & abstract;
    2. Java 8 使接口可以有 default 方法和 static 方法;
    3. default method and static method 可以是已存在的接口添加新的功能而不影响原有逻辑;
    4. static method 类似 default method, 只是实现者不能 override 这些方法;
    5. default method 值方法签名前加 default 关键字;

      default void myDefaultMethod(){ ... }
    6. 抽象类可以有构造函数, interface 不行. 接口侧重定义规范, 抽象类侧重实现整体, 细节留空;
    7. 一个类实现多个接口中如果有相同的 default 方法, 编译会报错, 需要在实现类中解决冲突;
  5. Stream API (java.util.stream)

    1. using streams we can perform various aggregate operations on the data returned from collections, arrays, Input/Output operations;
    2. Stream 可顺序也可以通过并行执行(Parallel execution)的方式(理论上,实际不一定)加快执行速度;

      1. stream.sequential()
      2. stream.parallel()
    3. 步骤: 1) 一次创建 Stream; 2) 0或多个中间操作; 3) 一次终止操作;
    4. 例子: Arrays.asList("line0", "line1").stream().filter(str->str.length()<6).count()
    5. 常见的操作:
      Intermediate operations:

      1. filter - Exclude all elements that don't match a Predicate.
      2. map - Perform a one-to-one transformation of elements using a Function.
      3. flatMap - Transform each element into zero or more elements by way of another Stream.
      4. peek - Perform some action on each element as it is encountered. Primarily useful for debugging.
      5. distinct - Exclude all duplicate elements according to their .equals behavior. This is a stateful operation.
      6. sorted - Ensure that stream elements in subsequent operations are encountered according to the order imposed by a Comparator. This is a stateful operation.
      7. limit - Ensure that subsequent operations only see up to a maximum number of elements. This is a stateful, short-circuiting operation.
      8. skip - Ensure that subsequent operations do not see the first n elements. This is a stateful operation.

      Terminal operations:

      1. forEach - Perform some action for each element in the stream.
      2. toArray - Dump the elements in the stream to an array.
      3. reduce - Combine the stream elements into one using a BinaryOperator.
      4. collect - Dump the elements in the stream into some container, such as a Collection or Map.
      5. min - Find the minimum element of the stream according to a Comparator.
      6. max - Find the maximum element of the stream according to a Comparator.
      7. count - Find the number of elements in the stream.
      8. anyMatch - Find out whether at least one of the elements in the stream matches a Predicate. This is a short-circuiting operation.
      9. allMatch - Find out whether every element in the stream matches a Predicate. This is a short-circuiting operation.
      10. noneMatch - Find out whether zero elements in the stream match a Predicate. This is a short-circuiting operation.
      11. findFirst - Find the first element in the stream. This is a short-circuiting operation.
      12. findAny - Find any element in the stream, which may be cheaper than findFirst for some streams. This is a short-circuiting operation.
    6. stream 可以 infinite;
    7. stream 处理可以短路(Short-circuiting), 不在执行剩下的, 对于无限 stream 最有用;
  6. Optional

    1. 改变编程习惯, 避免 NullPointerException - 原来直接用, 现在用各种方法;
    2. A container object which may or may not contain a non-null value;
    3. 创建: 1) Optional.empty(), 2) Optional.of(T value), 3) Optional.ofNullable(T value);
    4. 判断: isPresent();
    5. 获得: get(), 若null -> NoSuchElementException;
    6. 获得带 fallback 机制:

      1. 若空返回 other: orElse(T other)
      2. 若空执行函数接口返回值: orElseGet(Supplier<? extends T> other)
      3. 若空抛出特定异常: orElseThrow(Supplier<? extends X> exceptionSupplier) throws X
    7. 其它

      1. 非空并且满足predicate filter(Predicate<? super T> predicate);
      2. map(Function<? super T, ? extends U> mapper)
      3. flatMap(Function<? super T, Optional<U>> mapper)
  7. StringJoiner 类似 guava 的 Joiner, 前缀, 后缀, 分隔符.
  8. Arrays.parallelSort -> 多线程排序加速
  9. java.util.function 一些常见的函数

    1. Function<T, R> - take a T as input, return an R as ouput
    2. Predicate - take a T as input, return a boolean as output
    3. Consumer - take a T as input, perform some action and don't return anything
    4. Supplier - with nothing as input, return a T
    5. BinaryOperator - take two T's as input, return one T as output, useful for "reduce" operations
  10. java.time 包

refer:

  1. https://beginnersbook.com/2017/10/java-8-features-with-examples/
  2. https://www.techempower.com/blog/2013/03/26/everything-about-java-8/
  3. https://www.javatpoint.com/java-8-features
  4. https://www.oracle.com/technetwork/java/javase/8-whats-new-2157071.html

解决 Non-numeric value found - int expected 问题

在使用 btrace, 远程 debug 工具, JDK 自带小工具通过 agent 去连目标 Java 进程的时候, 有时候会遇到这个错误: Non-numeric value found - int expected. 我们明明给了一个目标进程的 int ID, 却报这个错误.

这里的原因是当时 client 使用的 Java 版本和目标进程的 Java 版本不一致造成的.

通过修改其中一个 Java 版本或者设置其中一个的 JAVA_HOME 变量, 使他们版本一致, 这个问题就解决了.

诊断由 modelmapper 导致的内存泄漏

发现一个 Java 应用 GC overhead 非常高, 查看 verbose GC log, 发现 heap 基本用完. 做了一个 heap dump, 发现其中一个 ClassLoader 管理着 327680 个类, 占用了 1.2G 的空间.

The classloader/component "org.springframework.boot.loader.LaunchedURLClassLoader @ 0x768800000" occupies 1,296,913,416 (77.87%) bytes. The memory is accumulated in one instance of "java.lang.Object[]" loaded by "<system class loader>".

仔细检查最新加入的这些类(一般Java 应用运行在稳定状态,很少有新的类被载入), 都有一些通用的模式:

  1. 类名类似于: com.tianxiaohui.MyClass$ByteBudddy$Ag2xax0, 前面的到 ByteBuddy 都是一样的, 后面全是类似 Hash 码的字符串, 看上去是一些代理子类;
  2. 类名类似于: org.modelmapper.internal.TypeMapImpl$Property@3f59d6c7, 都是基于 TypeMapImpl$Property的一些类.

通过以下 btrace 脚本, 在加入的 ClassLoader 的 classes 之前, 可以截获这些类, 并能查看到底哪个地方新加的这些类:

package test;

import static org.openjdk.btrace.core.BTraceUtils.println;

import org.openjdk.btrace.core.BTraceUtils;
import org.openjdk.btrace.core.BTraceUtils.Strings;
import org.openjdk.btrace.core.annotations.BTrace;
import org.openjdk.btrace.core.annotations.OnMethod;
import org.openjdk.btrace.core.annotations.ProbeClassName;
import org.openjdk.btrace.core.annotations.ProbeMethodName;

@BTrace
public class NewClassTracer {

    @OnMethod( clazz="/java\\.util\\.Vector/", method="/addElement/")
    public static void m(@ProbeClassName String probeClass, @ProbeMethodName String probeMethod, Object obj) {
            //print(Strings.strcat("entered ", probeClass));
            //println(Strings.strcat(".", probeMethod));
            println(Strings.strcat("new Class: ", Strings.str(obj)));
            BTraceUtils.jstack();
    }
}

通过上面的 btrace 脚本, 可以看到如下的 stacktrace:

java.util.Vector.addElement(Vector.java)
java.lang.ClassLoader.addClass(ClassLoader.java:263)
java.lang.ClassLoader.defineClass1(Native Method)
java.lang.ClassLoader.defineClass(ClassLoader.java:763)
sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.modelmapper.internal.bytebuddy.dynamic.loading.ClassInjector$UsingReflection$Dispatcher$Direct.defineClass(ClassInjector.java:604)
org.modelmapper.internal.bytebuddy.dynamic.loading.ClassInjector$UsingReflection.injectRaw(ClassInjector.java:235)
org.modelmapper.internal.bytebuddy.dynamic.loading.ClassInjector$AbstractBase.inject(ClassInjector.java:111)
org.modelmapper.internal.bytebuddy.dynamic.loading.ClassLoadingStrategy$Default$InjectionDispatcher.load(ClassLoadingStrategy.java:232)
org.modelmapper.internal.bytebuddy.dynamic.loading.ClassLoadingStrategy$Default.load(ClassLoadingStrategy.java:143)
org.modelmapper.internal.bytebuddy.dynamic.TypeResolutionStrategy$Passive.initialize(TypeResolutionStrategy.java:100)
org.modelmapper.internal.bytebuddy.dynamic.DynamicType$Default$Unloaded.load(DynamicType.java:5623)
org.modelmapper.internal.ProxyFactory.proxyFor(ProxyFactory.java:97)
org.modelmapper.internal.ProxyFactory.proxyFor(ProxyFactory.java:72)
org.modelmapper.internal.ReferenceMapExpressionImpl.map(ReferenceMapExpressionImpl.java:67)
org.modelmapper.internal.ConfigurableConditionExpressionImpl.map(ConfigurableConditionExpressionImpl.java:65)

那么诊断下来, 就是每次使用下面的代码的时候, 就创建一些新的类 (例子代码中 Man 和 Person 都是一个只有 name 字段的 POJO):

People p = new People();
p.setName("eric");

//type 1
ModelMapper modelMapper = new ModelMapper();
Man man = modelMapper.map(p, Man.class);
System.out.println(man);

//type 2
ModelMapper modelMapper2 = new ModelMapper();
modelMapper2.getConfiguration().setAmbiguityIgnored(true);

TypeMap<People, Man> typeMap = modelMapper2.createTypeMap(People.class, Man.class);
typeMap.addMappings(mapper -> {
    mapper.map(source -> source.getName(), Man::setName);
});

System.out.println(modelMapper2.map(p, Man.class));

原因在于每个 ModelMapper 实例都会管理自己的 Model, 每次都会创建一些新的类. 所以官方站点上明确说明:

Unless you need different mappings between the same types, then it’s
best to re-use the same ModelMapper instance.

一些其他人遇到类似的问题: https://github.com/modelmapper/modelmapper/issues/375

所以, 最好是这些 ModelMapper 都是 static final 的, 保证尽最大可能重用, 否则就会出现内存溢出问题.