Java:外部内存"VarHandleSegmentViewBase“错误，地址访问错误

Question

我正在尝试理解如何读取/写入一个简单结构的二进制编码版本，如下所示：

typedef struct Tuple {
    uint32_t length;
    uint8_t* data;
} Tuple;

我有以下代码，它可以正确地用Java将Tuple记录写入MemorySegment中的二进制数据。但是试图将数据读回失败--而一个简单的C程序可以很好地读取它。

Caused by: java.lang.IllegalArgumentException: Misaligned access at address: 16
    at java.base/java.lang.invoke.VarHandleSegmentViewBase.newIllegalArgumentExceptionForMisalignedAccess(VarHandleSegmentViewBase.java:57)
    at java.base/java.lang.invoke.VarHandleSegmentAsInts.offsetNoVMAlignCheck(VarHandleSegmentAsInts.java:100)
    at java.base/java.lang.invoke.VarHandleSegmentAsInts.get(VarHandleSegmentAsInts.java:111)
    at com.example.Tuple.deserialize(App.java:233)

下面我做错了什么？

record Tuple(int size, byte[] data) {

    public static ValueLayout.OfInt SIZE_FIELD = ValueLayout.JAVA_INT.withName("size");
    public static ValueLayout.OfAddress DATA_FIELD = ValueLayout.ADDRESS.withName("data").withBitAlignment(32);

    public static GroupLayout LAYOUT = MemoryLayout.structLayout(SIZE_FIELD, DATA_FIELD).withName("Tuple");

    public static VarHandle VH_SIZE = LAYOUT.varHandle(MemoryLayout.PathElement.groupElement("size"));
    public static VarHandle VH_DATA = LAYOUT.varHandle(MemoryLayout.PathElement.groupElement("data"));

    Tuple(byte[] data) {
        this(data.length, data);
    }

    public static Tuple deserialize(MemorySegment segment) {
        int size = (int) VH_SIZE.get(segment);
        byte[] data = segment
                .asSlice(SIZE_FIELD.byteSize())
                .toArray(ValueLayout.JAVA_BYTE);
        return new Tuple(size, data);
    }

    public int byteSize() {
        return (int) (size + ValueLayout.JAVA_INT.byteSize());
    }

    public void serialize(MemorySegment segment) {
        VH_SIZE.set(segment, size);
        segment
                .asSlice(ValueLayout.JAVA_INT.byteSize())
                .copyFrom(MemorySegment.ofArray(data));
    }
}

public static void main(String[] args) throws Exception {
    Tuple tuple = new Tuple("Hello".getBytes());

    File file = new File("tuple.bin");
    file.createNewFile();

    try (MemorySession session = MemorySession.openConfined()) {
        MemorySegment segment = session.allocate(tuple.byteSize() * 2L);
        tuple.serialize(segment);
        byte[] bytes = segment.toArray(ValueLayout.JAVA_BYTE);
        Files.write(file.toPath(), bytes);
    } catch (Exception e) {
        throw new RuntimeException(e);
    }

    // Now read the file back in
    try (MemorySession session = MemorySession.openConfined()) {
        byte[] bytes = Files.readAllBytes(Paths.get("tuple.bin"));
        MemorySegment segment = MemorySegment.ofArray(bytes);
        Tuple tuple2 = Tuple.deserialize(segment);
        System.out.println(tuple2);
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}

我使用下面的C程序来确认它在那里工作：

#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>

typedef struct Tuple {
    uint32_t length;
    uint8_t* data;
} Tuple;

const char* EXAMPLE_FILE = "tuple.bin";

int main() {
    FILE* file = fopen(EXAMPLE_FILE, "rb");
    if (file == NULL) {
        printf("Could not open file %s\n", EXAMPLE_FILE);
        return 1;
    }

    Tuple tuple;
    fread(&tuple.length, sizeof(uint32_t), 1, file);
    tuple.data = malloc(tuple.length);
    fread(tuple.data, sizeof(uint8_t), tuple.length, file);
    fclose(file);

    // Convert tuple data to string
    char* string = malloc(tuple.length + 1);
    for (uint32_t i = 0; i < tuple.length; i++) {
        string[i] = tuple.data[i];
    }
    string[tuple.length] = '\0';

    printf("Tuple size: %u bytes\n", tuple.length);
    printf("Tuple data: %s\n", string);

    return 0;
}

编辑：在使用pahole查看已编译的结构布局之后，似乎在Tuple字段之间插入了4个字节填充以进行对齐。

也许这个错误与此有关？

struct Tuple {
    uint32_t                   length;               /*     0     4 */

    /* XXX 4 bytes hole, try to pack */

    uint8_t *                  data;                 /*     8     8 */

    /* size: 16, cachelines: 1, members: 2 */
    /* sum members: 12, holes: 1, sum holes: 4 */
    /* last cacheline: 16 bytes */
};

​

​

​

​

Answer 1

您正在从byte[]读取带有对齐布局的值，但是byte[]的元素仅被假定/保证对齐为1字节(即不对齐)。因此，使用具有更大的对齐约束的布局(例如您正在使用的布局)读取/写入一个值将失败，出现类似的例外情况。

在编写数据的情况下，它可能有效，因为在这种情况下，您使用的是本机段。对于本机段，“真实”内存地址用于进行对齐检查，底层分配器(即malloc )通常返回16字节对齐内存区域。(不过，Java代码只要求1字节对齐。

如果您想直接从byte[]中读取，您可以通过将.withBitAlignment(8)应用于所有正在使用的布局来绕过异常，这将有效地关闭对齐检查。

Answer 2

我设法修复了它，我做了两个修改：

1. 我使用jextract工具生成与我类似的Java代码，以查看它的不同之处。它基本上是相同的，但是他们定义了这样的C互操作类型：

class Constants {
    static final ValueLayout.OfBoolean C_BOOL_LAYOUT = ValueLayout.JAVA_BOOLEAN;
    static final ValueLayout.OfByte C_CHAR_LAYOUT = ValueLayout.JAVA_BYTE;
    static final ValueLayout.OfShort C_SHORT_LAYOUT = ValueLayout.JAVA_SHORT.withBitAlignment(16);
    static final ValueLayout.OfInt C_INT_LAYOUT = ValueLayout.JAVA_INT.withBitAlignment(32);
    static final ValueLayout.OfInt C_LONG_LAYOUT = ValueLayout.JAVA_INT.withBitAlignment(32);
    static final ValueLayout.OfLong C_LONG_LONG_LAYOUT = ValueLayout.JAVA_LONG.withBitAlignment(64);
    static final ValueLayout.OfFloat C_FLOAT_LAYOUT = ValueLayout.JAVA_FLOAT.withBitAlignment(32);
    static final ValueLayout.OfDouble C_DOUBLE_LAYOUT = ValueLayout.JAVA_DOUBLE.withBitAlignment(64);
    static final ValueLayout.OfAddress C_POINTER_LAYOUT = ValueLayout.ADDRESS.withBitAlignment(64);
}

我注意到它在字段之间插入了32字节的填充。因此，我将Tuple布局的定义更改为：

public static ValueLayout.OfInt SIZE_FIELD = Constants.C_LONG_LAYOUT.withName("size");
public static ValueLayout.OfAddress DATA_FIELD = Constants.C_POINTER_LAYOUT.withName("data");

public static GroupLayout LAYOUT = MemoryLayout.structLayout(
        SIZE_FIELD,
        MemoryLayout.paddingLayout(32),
        DATA_FIELD
).withName("Tuple");

1. 我没有尝试将内存段从MemorySegment.ofArray(bytes)传递给.deserialize()，而是首先分配内存，然后将字节内存复制到内存中：

// Now read the file back in
try (MemorySession session = MemorySession.openConfined()) {
    byte[] bytes = Files.readAllBytes(Paths.get("tuple.bin"));

    MemorySegment segment = session.allocate(Tuple.LAYOUT);
    segment.copyFrom(MemorySegment.ofArray(bytes));

    Tuple tuple2 = Tuple.deserialize(segment);
    System.out.println(tuple2);
    System.out.println(new String(tuple2.data()));
} catch (Exception e) {
    throw new RuntimeException(e);
}