首页

分片上传

    本SDK提供CreateMultipartUploadCommand,UploadPartCommand,CompleteMultipartUploadCommand,AbortMultipartUploadCommand这四个类共同实现分片上传，完整代码详见 Github 。 CreateMultipartUploadCommand调用的 S3 API 为 CreateMultipartUpload, 具体参见CreateMultipartUpload API 文档。 UploadPartCommand调用的 S3 API 为 UploadPart, 具体参见UploadPart API 文档。 CompleteMultipartUploadCommand调用的 S3 API 为 CompleteMultipartUpload, 具体参见CompleteMultipartUpload API 文档。 AbortMultipartUploadCommand调用的 S3 API 为 AbortMultipartUpload, 具体参见AbortMultipartUpload API 文档。

    分片上传流程

    在US3中，一个完整的分片上传分为初始化分片 -> 上传分片 -> 完成分片(中止分片)三个阶段。

    参数说明

    • Bucket： 文件上传后所在的存储空间
    • Key: 文件上传后在存储空间里的名称
    • UploadId: 用于关联所有分片的唯一标识符
    • PartNumber: 当前上传的分片编号
    • Body: 要上传的文件分片的内容
    • ContentMD5: 指定分片数据的 MD5 校验值

    示例

    执行该示例前请确保配置文件的正确性
    以下代码段需要在上下文中运行

    1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91

    const {
    CreateMultipartUploadCommand,
    UploadPartCommand,
    CompleteMultipartUploadCommand,
    AbortMultipartUploadCommand
} = require("@aws-sdk/client-s3");
const s3 = require('../s3client');
const fs = require('fs');
const crypto = require('crypto');

// 计算 MD5

const calculateMD5 = (data) => {
return crypto.createHash('md5').update(data).digest('base64');
};

const sliceUpload = async (bucketName, key, filePath) => {
const fileBuffer = fs.readFileSync(filePath);
const partSize = 8 * 1024 * 1024; // 固定8MB

const numParts = Math.ceil(fileBuffer.length / partSize);

    let uploadId;

    try {
        // 创建分片上传任务

        const createMultipartUploadResponse = await s3.send(
            new CreateMultipartUploadCommand({
                Bucket: bucketName,
                Key: key,
            })
        );

        uploadId = createMultipartUploadResponse.UploadId;
        const uploadedParts = [];

        for (let partNumber = 1; partNumber <= numParts; partNumber++) {
            const start = (partNumber - 1) * partSize;
            const end = Math.min(start + partSize, fileBuffer.length);
            const partBuffer = fileBuffer.subarray(start, end);
            const md5Hash = calculateMD5(partBuffer);
            const uploadPartCommand = new UploadPartCommand({
                Bucket: bucketName,
                Key: key,
                UploadId: uploadId,
                PartNumber: partNumber,
                Body: partBuffer,
                ContentMD5: md5Hash,
            });

            const uploadPartResponse = await s3.send(uploadPartCommand);
            console.log(`Part ${partNumber} uploaded successfully`);
            uploadedParts.push({
                ETag: uploadPartResponse.ETag,
                PartNumber: partNumber,
            });
        }

        // 验证分片

        uploadedParts.forEach((part) => {
            if (!part.ETag) {
                throw new Error(`Part ${part.PartNumber} failed to upload.`);
            }
        });

        // 完成分片上传

        await s3.send(
            new CompleteMultipartUploadCommand({
                Bucket: bucketName,
                Key: key,
                UploadId: uploadId,
                MultipartUpload: {
                    Parts: uploadedParts,
                },
            })
        );

        console.log(`Successfully uploaded ${key} to ${bucketName}`);
    } catch (error) {
        console.error("Error uploading file:", error);

        if (uploadId) {
            // 中止上传

            await s3.send(
                new AbortMultipartUploadCommand({
                    Bucket: bucketName,
                    Key: key,
                    UploadId: uploadId,
                })
            );
        }
    }
};

    在 Example/ 目录中运行以下命令执行该示例

    1

    $ node SliceUpload.js <bucketName> <keyName> <filePath>