Contact~

Based on .NET Core + jQuery to implement file breakpoint chunk upload

发布： 3/30/2023 03:06

更新： 6/29/2023 06:53

阅读： 337

字数：18618

This project is implemented using .NET Core and jQuery for file chunked upload. It has not been tested because the blogger does not have such large files to test with. Currently, uploading files around 2GB in size works without any issues.

Preface

Based on .NET Core + JQuery Implement File Breakpoint Fragment Upload

Technologies Used

Redis caching technology JQuery AJAX request technology Why Redis is used will be explained later; let's leave it as a mystery for now.

Page Screenshot

Shard 03

NuGet Package

Microsoft.Extensions.Caching.StackExchangeRedis Zack.ASPNETCore Yang Zekun encapsulated Redis operation package How is multipart upload performed? When implementing the code, we need to understand why files are uploaded in chunks. Can't I just upload them directly? When using websites like Bilibili and Kuaishou to upload videos, you can see that if the file is interrupted, the previously uploaded parts will be quickly re-uploaded. This is the benefit of chunked uploads. If an interruption occurs, I only need to upload the chunks that have not been completed. When uploading a large file, the user might lose their internet connection or experience other issues causing the upload to fail. But would it be reasonable to have to re-upload several gigabytes of data? Of course not. Specifically, the principle of uploading files in chunks is as follows: The client splits large files into several small file chunks and generates a unique identifier for each chunk to facilitate subsequent merging operations. 2. The client uploads each small file chunk to the server and sends its identifier and other necessary information to the server. 3. After the server receives each small file chunk, it saves it in a temporary folder and returns an identifier to the client for subsequent merging operations. 4. The client sends the identifiers of all small file chunks to the server and requests that the server merge these small file chunks into a complete file. 5. After receiving the client's request, the server merges all small file chunks in order of their identifiers and saves the merged file at the specified location. 6. After the client receives the server's response, it confirms that the file upload was successful. In general, the principle of multipart upload is to divide a large file into several small chunks, upload each chunk separately to the server, and then merge these small chunks into a complete file. Start implementing the code after understanding the principles.

Backend Implementation

builder.Services.AddScoped<IDistributedCacheHelper, DistributedCacheHelper>();
//Register Redis service
```csharp
builder.Services.AddStackExchangeRedisCache(options =>

Please provide the content you would like translated to English. string connStr = builder.Configuration.GetSection("Redis").Value; string password = builder.Configuration.GetSection("RedisPassword").Value; //Redis server address options.Configuration = $",password="; }); Sure, please provide the content you would like translated to English. Configure Redis-related information in appsettings.json Sure, please provide the content you would like translated to English. "Redis": "redis address", "RedisPassword": "password" Sure, please provide the content you would like translated to English.

Implementation of Saving Files

Inject into the controller

private readonly IWebHostEnvironment _environment;
private readonly IDistributedCacheHelper _distributedCache;
```csharp
public UpLoadController(IDistributedCacheHelper distributedCache, IWebHostEnvironment environment)

Please provide the content you would like translated to English. _distributedCache = distributedCache; _environment = environment; } Please provide the content you would like translated to English. Retrieve the file name from Redis

string GetTmpChunkDir(string fileName)
Please provide the content you would like translated to English.
```javascript
var s = _distributedCache.GetOrCreate<string>(fileName, (e) =>

Please provide the content you would like translated to English. //Sliding expiration time // e.SlidingExpiration = TimeSpan.FromSeconds(1800);

//return Encoding.Default.GetBytes(Guid.NewGuid().ToString("N"));

return fileName.Split('.')[0]; }, 1800); if (s != null) return fileName.Split('.')[0]; return ""; } Sure, please provide the content you would like translated to English. Implement the save file method

/// <summary>
Save file
/// </summary>
/// <param name="file">File</param>
/// <param name="fileName">File name</param>
/// <param name="chunkIndex">File chunk</param>
/// <param name="chunkCount">Number of chunks</param>
/// <returns></returns>
```csharp
public async Task<JsonResult> SaveFile(IFormFile file, string fileName, int chunkIndex, int chunkCount)

Please provide the content you would like translated to English. try Please provide the content you would like translated to English. //Description is empty if (file.Length == 0) Please provide the content you would like translated to English. return Json(new Please provide the content you would like translated to English. success = false, mas = "File is empty!!!" }); } if (chunkIndex == 0) Please provide the content you would like translated to English. When uploading for the first time, generate a random ID to serve as a temporary folder for saving blocks. // Save the filename to Redis with a time of s

_distributedCache.GetOrCreate(fileName, (e) =>

Please provide the content you would like translated to English. //Sliding expiration time // e.SlidingExpiration = TimeSpan.FromSeconds(1800);

// return Encoding.Default.GetBytes(Guid.NewGuid().ToString("N"));

return fileName.Split('.')[0]; }, 1800); } if (!Directory.Exists(GetFilePath())) Directory.CreateDirectory(GetFilePath());

var fullChunkDir = GetFilePath() + dirSeparator + GetTmpChunkDir(fileName);

if (!Directory.Exists(fullChunkDir)) Directory.CreateDirectory(fullChunkDir);

var blog = file.FileName;

var newFileName = blog + chunkIndex + Path.GetExtension(fileName);

var filePath = fullChunkDir + Path.DirectorySeparatorChar + newFileName;

//If the file block does not exist, save it; otherwise, skip directly. if (!System.IO.File.Exists(filePath)) Please provide the content you would like translated to English. //Save file block using (var stream = new FileStream(filePath, FileMode.Create)) Please provide the content you would like translated to English. await file.CopyToAsync(stream); } } All blocks uploaded if (chunkIndex == chunkCount - 1) Please provide the content you would like translated to English. //Can also merge here, if merged here, no need to call ajax to combine CombineChunkFile //CombineChunkFile(fileName); }

var obj = new

Please provide the content you would like translated to English. success = true, date = DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss"), newFileName, originalFileName = fileName, size = file.Length, nextIndex = chunkIndex + 1, }; return Json(obj); } catch (Exception ex) Please provide the content you would like translated. return Json(new Please provide the content you would like translated to English. success = false, msg = ex.Message, }); } } Sure, please provide the content you would like translated to English.

Explain the Key Code - Redis Part

Of course, it can also be placed in the session. I will not demonstrate that here. This is storing the filename in Redis as a unique key value, of course it would be best to use

Encoding.Default.GetBytes(Guid.NewGuid().ToString("N"));

I used the file name to generate a random ID. I originally wrote this for sharing files with my roommate during computer class at school, so I didn't consider much and just did it based on my own needs. When uploading a file for the first time, Redis will save the filename. If the filename already exists in Redis, subsequent file chunks can be directly placed under that filename.

_distributedCache.GetOrCreate(fileName, (e) =>
Please provide the content you would like translated to English.
//Slide expiration time
// e.SlidingExpiration = TimeSpan.FromSeconds(1800);
```csharp
//return Encoding.Default.GetBytes(Guid.NewGuid().ToString("N"));

return fileName.Split('.')[0]; }, 1800); Please provide the content you would like translated into English.

Merge Files Method

//Directory separator, compatible with different systems
```csharp
private static readonly char dirSeparator = Path.DirectorySeparatorChar;

Please provide the content you would like translated into English.

//Get the storage path of the file
//Folder for saving
private string GetFilePath()
Please provide the content you would like translated to English.
return Path.Combine(_environment.WebRootPath, "UploadFolder");
}
Sure, please provide the content you would like translated to English.
```csharp
```csharp
public async Task<JsonResult> CombineChunkFile(string fileName)

Please provide the content you would like translated to English. try Please provide the content you would like translated to English. return await Task.Run(() => Please provide the content you would like translated to English. //Get the unique id value of the file, which is the file name

var tmpDir = GetTmpChunkDir(fileName);

//Find the directory where the file blocks are stored var fullChunkDir = GetFilePath() + dirSeparator + tmpDir; //Start Time

var beginTime = DateTime.Now;

//new file name

var newFileName = tmpDir + Path.GetExtension(fileName);

var destFile = GetFilePath() + dirSeparator + newFileName;

//Get all file chunks in the temporary folder and sort them

var files = Directory.GetFiles(fullChunkDir).OrderBy(x => x.Length).ThenBy(x => x).ToList();

//Combine file blocks into one file using (var destStream = System.IO.File.OpenWrite(destFile)) Please provide the content you want to translate. files.forEach(chunk => Please provide the content you would like translated to English.

using (var chunkStream = System.IO.File.OpenRead(chunk))

Please provide the content you would like translated to English. chunkStream.CopyTo(destStream); } System.IO.File.Delete(chunk); }); Directory.Delete(fullChunkDir); } //End time

var totalTime = DateTime.Now.Subtract(beginTime).TotalSeconds;

return Json(new Please provide the content you would like translated to English. success = true, destFile = destFile.Replace('\', '/'); msg = $"Merge completed! s", }); });

} catch (Exception ex)

Please provide the content you would like translated to English. return Json(new Please provide the content you would like translated to English. success = false, msg = ex.Message, }); } finally Please provide the content you would like translated to English. _distributedCache.Remove(fileName); } } Sure, please provide the content you would like translated to English.

Frontend Implementation

Principles

The principle is to obtain the file, then slice it, and recursively request the backend interface to save the file through the slices. ![/media/blog/b19252e475cf9ff2/slice02.png] First, include jQuery.

<script src="~/lib/jquery/dist/jquery.min.js"></script>

Then write an upload page randomly.

<div class="dropzone" id="dropzone"></div>
Drag and drop files here to upload.
Or<br>
```html
<input type="file" id="file1">

Sure, please provide the content you would like translated. ## CSS Implementation Make the page slightly more readable. ``` css Sure, please provide the content you would like me to translate. ## jQuery Code Implementation ```js ```html Sure, please provide the content you would like translated to English. ## Merge File Request ```javascript var isLastChunk = chunkIndex === chunkCount - 1; ``` When isLastChunk is true, perform file merging; in this case, there will be no further request to save the file. # Summary The principle of uploading files in chunks is very simple. Based on the principle, implementing the code and gradually exploring it will help you master it quickly. Of course, this article has many poorly written parts that can be pointed out. After all, the blogger is still a student who needs to continue learning. There are comments with questions; I will reply when I see them. # References - https://www.bilibili.com/video/BV1rB4y1Q7dF/?share_source=copy_web&vd_source=fce337a51d11a67781404c67ec0b5084

_EOF_

Based on .NET Core + jQuery to implement file breakpoint chunk upload

本文链接： https://starx.cc/blog/post/b19252e475cf9ff2.html

关于博主： 欢迎扫码加好友，交流技术 & 分享经验！

「博主」专注于互联网热门新技术探索与开发实践，包括架构设计、机器学习与数据分析算法、移动端开发、Linux~ 我都不会，欢迎一起探讨技术，分享学习实践经验。

随机推荐文章

讨论区

共 {{comments.length}} 条讨论

良言一句三冬暖，恶语伤人六月寒。

@{{replyComment.anonymousUser.name}}

刷新

重置

发表

首次发言：{{comment.anonymousUser.createdTime}}
网站地址：{{comment.anonymousUser.url}}

@{{comment.replyUser}} {{comment.content}}

Contact~

Preface

Based on .NET Core + JQuery Implement File Breakpoint Fragment Upload

Technologies Used

Page Screenshot

NuGet Package

Backend Implementation

Implementation of Saving Files

Explain the Key Code - Redis Part

Merge Files Method

Frontend Implementation

Principles

Based on .NET Core + jQuery to implement file breakpoint chunk upload

讨论区

{{comment.anonymousUser.name}}

{{comment.anonymousUser.name}}