经验首页 前端设计 程序设计 Java相关 移动开发 数据库/运维 软件/图像 大数据/云计算 其他经验
当前位置:技术经验 » 其他 » 职业生涯 » 查看文章
造轮子之文件管理
来源:cnblogs  作者:饭勺oO  时间:2023/10/25 10:13:40  对本文有异议

前面我们完成了设置管理,接下来正好配合设置管理来实现文件管理功能。
文件管理自然包括文件上传,下载以及文件存储功能。设计要求可以支持扩展多种存储服务,如本地文件,云存储等等。

数据库设计

首先当然是我们的数据库表设计,用于管理文件。创建一个文件信息存储表。

  1. using Wheel.Domain.Common;
  2. using Wheel.Enums;
  3. namespace Wheel.Domain.FileStorages
  4. {
  5. /// <summary>
  6. /// 文件信息存储表
  7. /// </summary>
  8. public class FileStorage : Entity, IHasCreationTime
  9. {
  10. /// <summary>
  11. /// 文件名
  12. /// </summary>
  13. public string FileName { get; set; }
  14. /// <summary>
  15. /// 文件类型ContentType
  16. /// </summary>
  17. public string ContentType { get; set; }
  18. /// <summary>
  19. /// 文件类型
  20. /// </summary>
  21. public FileStorageType FileStorageType { get; set; }
  22. /// <summary>
  23. /// 大小
  24. /// </summary>
  25. public long Size { get; set; }
  26. /// <summary>
  27. /// 存储路径
  28. /// </summary>
  29. public string Path { get; set; }
  30. /// <summary>
  31. /// 创建时间
  32. /// </summary>
  33. public DateTimeOffset CreationTime { get; set; }
  34. /// <summary>
  35. /// 存储类型
  36. /// </summary>
  37. public string Provider { get; set; }
  38. }
  39. }
  1. namespace Wheel.Enums
  2. {
  3. public enum FileStorageType
  4. {
  5. /// <summary>
  6. /// 普通文件
  7. /// </summary>
  8. File = 0,
  9. /// <summary>
  10. /// 图片
  11. /// </summary>
  12. Image = 1,
  13. /// <summary>
  14. /// 视频
  15. /// </summary>
  16. Video = 2,
  17. /// <summary>
  18. /// 音频
  19. /// </summary>
  20. Audio = 3,
  21. /// <summary>
  22. /// 文本类型
  23. /// </summary>
  24. Text = 4,
  25. }
  26. }

FileStorageType是对ContentType类型的包装。后面可根据需求再加上细分类型。

  1. using Wheel.Enums;
  2. namespace Wheel.Domain.FileStorages
  3. {
  4. public static class FileStorageTypeChecker
  5. {
  6. public static FileStorageType CheckFileType(string contentType)
  7. {
  8. return contentType switch
  9. {
  10. var _ when contentType.StartsWith("audio") => FileStorageType.Audio,
  11. var _ when contentType.StartsWith("image") => FileStorageType.Image,
  12. var _ when contentType.StartsWith("text") => FileStorageType.Text,
  13. var _ when contentType.StartsWith("video") => FileStorageType.Video,
  14. _ => FileStorageType.File
  15. };
  16. }
  17. }
  18. }

Provider对应不同的存储服务。如Minio等。

修改DbContext

在DbContext中添加代码:

  1. #region FileStorage
  2. public DbSet<FileStorage> FileStorages { get; set; }
  3. #endregion
  4. protected override void OnModelCreating(ModelBuilder builder)
  5. {
  6. base.OnModelCreating(builder);
  7. ConfigureIdentity(builder);
  8. ConfigureLocalization(builder);
  9. ConfigurePermissionGrants(builder);
  10. ConfigureMenus(builder);
  11. ConfigureSettings(builder);
  12. ConfigureFileStorage(builder);
  13. }
  14. void ConfigureFileStorage(ModelBuilder builder)
  15. {
  16. builder.Entity<FileStorage>(b =>
  17. {
  18. b.HasKey(o => o.Id);
  19. b.Property(o => o.FileName).HasMaxLength(256);
  20. b.Property(o => o.Path).HasMaxLength(256);
  21. b.Property(o => o.ContentType).HasMaxLength(32);
  22. b.Property(o => o.Provider).HasMaxLength(32);
  23. });
  24. }

然后执行数据库迁移操作即可完成表创建。

FileStorageProvider

接下来就是实现我们的文件存储的Provider,首先创建一个IFileStorageProvider基础接口。

  1. using Wheel.DependencyInjection;
  2. namespace Wheel.FileStorages
  3. {
  4. public interface IFileStorageProvider : ITransientDependency
  5. {
  6. string Name { get; }
  7. Task<UploadFileResult> Upload(UploadFileArgs uploadFileArgs, CancellationToken cancellationToken = default);
  8. Task<DownFileResult> Download(DownloadFileArgs downloadFileArgs, CancellationToken cancellationToken = default);
  9. Task<object> GetClient();
  10. void ConfigureClient<T>(Action<T> configure);
  11. }
  12. }

提供定义名称,上传下载,以及获取Provider的Client和配置Provider中的Client的方法。

FileProviderSettingDefinition

既然要对接各种存储服务,那么当然少不了对接的配置,那么我们就基于前面设置管理。添加一个FileProviderSettingDefinition

  1. using Wheel.Enums;
  2. namespace Wheel.Settings.FileProvider
  3. {
  4. public class FileProviderSettingDefinition : ISettingDefinition
  5. {
  6. public string GroupName => "FileProvider";
  7. public SettingScope SettingScope => SettingScope.Global;
  8. public Dictionary<string, SettingValueParams> Define()
  9. {
  10. return new Dictionary<string, SettingValueParams>
  11. {
  12. { "Minio.Endpoint", new(SettingValueType.String, "127.0.0.1:9000") },
  13. { "Minio.AccessKey", new(SettingValueType.String, "2QgNxo11uxgULRvkrdaT") },
  14. { "Minio.SecretKey", new(SettingValueType.String, "NvzXnh81UMwEcvLJc8BslA1GA0j0sCq0aXRgHSRJ") },
  15. { "Minio.Region", new(SettingValueType.String) },
  16. { "Minio.SessionToken", new(SettingValueType.String) }
  17. };
  18. }
  19. }
  20. }

这里我暂时只实现对接Minio,所以只加上Minio的配置。

MinioFileStorageProvider

接下来实现一个MinioFileStorageProvider

  1. using Minio;
  2. using Minio.DataModel.Args;
  3. using Minio.Exceptions;
  4. using Wheel.Settings;
  5. namespace Wheel.FileStorages.Providers
  6. {
  7. public class MinioFileStorageProvider : IFileStorageProvider
  8. {
  9. private readonly ISettingProvider _settingProvider;
  10. private readonly ILogger<MinioFileStorageProvider> _logger;
  11. public MinioFileStorageProvider(ISettingProvider settingProvider, ILogger<MinioFileStorageProvider> logger)
  12. {
  13. _settingProvider = settingProvider;
  14. _logger = logger;
  15. }
  16. public string Name => "Minio";
  17. internal Action<IMinioClient>? Configure { get; private set; }
  18. public async Task<UploadFileResult> Upload(UploadFileArgs uploadFileArgs, CancellationToken cancellationToken = default)
  19. {
  20. var client = await GetMinioClient();
  21. try
  22. {
  23. // Make a bucket on the server, if not already present.
  24. var beArgs = new BucketExistsArgs()
  25. .WithBucket(uploadFileArgs.BucketName);
  26. bool found = await client.BucketExistsAsync(beArgs, cancellationToken).ConfigureAwait(false);
  27. if (!found)
  28. {
  29. var mbArgs = new MakeBucketArgs()
  30. .WithBucket(uploadFileArgs.BucketName);
  31. await client.MakeBucketAsync(mbArgs, cancellationToken).ConfigureAwait(false);
  32. }
  33. // Upload a file to bucket.
  34. var putObjectArgs = new PutObjectArgs()
  35. .WithBucket(uploadFileArgs.BucketName)
  36. .WithObject(uploadFileArgs.FileName)
  37. .WithStreamData(uploadFileArgs.FileStream)
  38. .WithObjectSize(uploadFileArgs.FileStream.Length)
  39. .WithContentType(uploadFileArgs.ContentType);
  40. await client.PutObjectAsync(putObjectArgs, cancellationToken).ConfigureAwait(false);
  41. var path = BuildPath(uploadFileArgs.BucketName, uploadFileArgs.FileName);
  42. _logger.LogInformation("Successfully Uploaded " + path);
  43. return new UploadFileResult { FilePath = path, Success = true };
  44. }
  45. catch (MinioException e)
  46. {
  47. _logger.LogError("File Upload Error: {0}", e.Message);
  48. return new UploadFileResult { Success = false };
  49. }
  50. }
  51. public async Task<DownFileResult> Download(DownloadFileArgs downloadFileArgs, CancellationToken cancellationToken = default)
  52. {
  53. var client = await GetMinioClient();
  54. try
  55. {
  56. var stream = new MemoryStream();
  57. var args = downloadFileArgs.Path.Split("/");
  58. var getObjectArgs = new GetObjectArgs()
  59. .WithBucket(args[0])
  60. .WithObject(downloadFileArgs.Path.RemovePreFix($"{args[0]}/"))
  61. .WithCallbackStream(fs => fs.CopyTo(stream))
  62. ;
  63. var response = await client.GetObjectAsync(getObjectArgs, cancellationToken).ConfigureAwait(false);
  64. _logger.LogInformation("Successfully Download " + downloadFileArgs.Path);
  65. stream.Position = 0;
  66. return new DownFileResult { Stream = stream, Success = true, FileName = response.ObjectName, ContentType = response.ContentType };
  67. }
  68. catch (MinioException e)
  69. {
  70. _logger.LogError("File Download Error: {0}", e.Message);
  71. return new DownFileResult { Success = false };
  72. }
  73. }
  74. public async Task<object> GetClient()
  75. {
  76. return await GetMinioClient();
  77. }
  78. public void ConfigureClient<T>(Action<T> configure)
  79. {
  80. if (typeof(T) == typeof(IMinioClient))
  81. Configure = configure as Action<IMinioClient>;
  82. else
  83. throw new Exception("MinioFileProvider ConfigureClient Only Can Configure Type With IMinioClient");
  84. }
  85. private async Task<IMinioClient> GetMinioClient()
  86. {
  87. var minioSetting = await GetSettings();
  88. var client = new MinioClient()
  89. .WithHttpClient(new HttpClient())
  90. .WithEndpoint(minioSetting["Endpoint"])
  91. .WithCredentials(minioSetting["AccessKey"], minioSetting["SecretKey"])
  92. .WithSessionToken(minioSetting["SessionToken"]);
  93. if (!string.IsNullOrWhiteSpace(minioSetting["Region"]))
  94. {
  95. client.WithRegion(minioSetting["Region"]);
  96. }
  97. if (Configure != null)
  98. {
  99. Configure.Invoke(client);
  100. }
  101. return client;
  102. }
  103. private async Task<Dictionary<string, string>> GetSettings()
  104. {
  105. var settings = await _settingProvider.GetGolbalSettings("FileProvider");
  106. return settings.Where(a => a.Key.StartsWith("Minio")).ToDictionary(a => a.Key.RemovePreFix("Minio."), a => a.Value);
  107. }
  108. private string BuildPath(string bucketName, string fileName)
  109. {
  110. return string.Join('/', bucketName, fileName);
  111. }
  112. }
  113. }

这里定义MinioFileStorageProvider的Name是Minio用作标识。
Upload和Download则是正常的使用MinioClient的上传下载操作。
GetClient()返回一个MinioClient实例,用于方便做其他“骚操作”。
ConfigureClient则是用来配置MinioClient实例,代码约定限制只支持IMinioClient的类型。
GetSettings则是从SettingProvider中获取Minio的配置信息。

FileStorageManageAppService

基础的对接搭好了,现在我们来实现我们的业务功能。很简单,就三个功能,上传下载,分页查询。

  1. using Wheel.Core.Dto;
  2. using Wheel.DependencyInjection;
  3. using Wheel.Services.FileStorageManage.Dtos;
  4. namespace Wheel.Services.FileStorageManage
  5. {
  6. public interface IFileStorageManageAppService : ITransientDependency
  7. {
  8. Task<Page<FileStorageDto>> GetFileStoragePageList(FileStoragePageRequest request);
  9. Task<R<List<FileStorageDto>>> UploadFiles(UploadFileDto uploadFileDto);
  10. Task<R<DownloadFileResonse>> DownloadFile(long id);
  11. }
  12. }
  1. using Wheel.Const;
  2. using Wheel.Core.Dto;
  3. using Wheel.Core.Exceptions;
  4. using Wheel.Domain;
  5. using Wheel.Domain.FileStorages;
  6. using Wheel.Enums;
  7. using Wheel.FileStorages;
  8. using Wheel.Services.FileStorageManage.Dtos;
  9. using Path = System.IO.Path;
  10. namespace Wheel.Services.FileStorageManage
  11. {
  12. public class FileStorageManageAppService : WheelServiceBase, IFileStorageManageAppService
  13. {
  14. private readonly IBasicRepository<FileStorage, long> _fileStorageRepository;
  15. public FileStorageManageAppService(IBasicRepository<FileStorage, long> fileStorageRepository)
  16. {
  17. _fileStorageRepository = fileStorageRepository;
  18. }
  19. public async Task<Page<FileStorageDto>> GetFileStoragePageList(FileStoragePageRequest request)
  20. {
  21. var (items, total) = await _fileStorageRepository.GetPageListAsync(
  22. _fileStorageRepository.BuildPredicate(
  23. (!string.IsNullOrWhiteSpace(request.FileName), f => f.FileName.Contains(request.FileName!)),
  24. (!string.IsNullOrWhiteSpace(request.ContentType), f => f.ContentType.Equals(request.ContentType)),
  25. (!string.IsNullOrWhiteSpace(request.Path), f => f.Path.StartsWith(request.Path!)),
  26. (!string.IsNullOrWhiteSpace(request.Provider), f => f.Provider.Equals(request.Provider)),
  27. (request.FileStorageType.HasValue, f => f.FileStorageType.Equals(request.FileStorageType))
  28. ),
  29. (request.PageIndex -1) * request.PageSize,
  30. request.PageSize,
  31. request.OrderBy
  32. );
  33. return new Page<FileStorageDto>(Mapper.Map<List<FileStorageDto>>(items), total);
  34. }
  35. public async Task<R<List<FileStorageDto>>> UploadFiles(UploadFileDto uploadFileDto)
  36. {
  37. var files = uploadFileDto.Files;
  38. if (files.Count == 0)
  39. return new R<List<FileStorageDto>>(new());
  40. IFileStorageProvider? fileStorageProvider = null;
  41. var fileStorageProviders = ServiceProvider.GetServices<IFileStorageProvider>();
  42. if (string.IsNullOrWhiteSpace(uploadFileDto.Provider))
  43. {
  44. fileStorageProvider = fileStorageProviders.First();
  45. }
  46. else
  47. {
  48. fileStorageProvider = fileStorageProviders.First(a => a.Name == uploadFileDto.Provider);
  49. }
  50. var fileStorages = new List<FileStorage>();
  51. foreach (var file in files)
  52. {
  53. var fileName = uploadFileDto.Cover ? file.FileName : $"{Path.GetFileNameWithoutExtension(file.FileName)}-{SnowflakeIdGenerator.Create()}{Path.GetExtension(file.FileName)}";
  54. var fileStream = file.OpenReadStream();
  55. var fileStorageType = FileStorageTypeChecker.CheckFileType(file.ContentType);
  56. var uploadFileArgs = new UploadFileArgs
  57. {
  58. BucketName = fileStorageType switch
  59. {
  60. FileStorageType.Image => "images",
  61. FileStorageType.Video => "videos",
  62. FileStorageType.Audio => "audios",
  63. FileStorageType.Text => "texts",
  64. _ => "files"
  65. },
  66. ContentType = file.ContentType,
  67. FileName = fileName,
  68. FileStream = fileStream
  69. };
  70. var uploadFileResult = await fileStorageProvider.Upload(uploadFileArgs);
  71. if (uploadFileResult.Success)
  72. {
  73. var fileStorage = await _fileStorageRepository.InsertAsync(new FileStorage
  74. {
  75. Id = SnowflakeIdGenerator.Create(),
  76. ContentType = file.ContentType,
  77. FileName = file.FileName,
  78. FileStorageType = fileStorageType,
  79. Path = uploadFileResult.FilePath,
  80. Provider = fileStorageProvider.Name,
  81. Size = fileStream.Length
  82. });
  83. await _fileStorageRepository.SaveChangeAsync();
  84. fileStorages.Add(fileStorage);
  85. }
  86. }
  87. return new R<List<FileStorageDto>>(Mapper.Map<List<FileStorageDto>>(fileStorages));
  88. }
  89. public async Task<R<DownloadFileResonse>> DownloadFile(long id)
  90. {
  91. var fileStorage = await _fileStorageRepository.FindAsync(id);
  92. if(fileStorage == null)
  93. {
  94. throw new BusinessException(ErrorCode.FileNotExist, "FileNotExist")
  95. .WithMessageDataData(id.ToString());
  96. }
  97. var fileStorageProvider = ServiceProvider.GetServices<IFileStorageProvider>().First(a=>a.Name == fileStorage.Provider);
  98. var downloadResult = await fileStorageProvider.Download(new DownloadFileArgs { Path = fileStorage.Path });
  99. if (downloadResult.Success)
  100. {
  101. return new R<DownloadFileResonse>(new DownloadFileResonse { ContentType = downloadResult.ContentType, FileName = downloadResult.FileName, Stream = downloadResult.Stream });
  102. }
  103. else
  104. {
  105. throw new BusinessException(ErrorCode.FileDownloadFail, "FileDownloadFail")
  106. .WithMessageDataData(id.ToString());
  107. }
  108. }
  109. }
  110. }

UploadFiles时如果没有指定Provider则默认取依赖注入第一个Provider,如果指定则取Provider。

  1. using Microsoft.AspNetCore.Mvc;
  2. namespace Wheel.Services.FileStorageManage.Dtos
  3. {
  4. public class UploadFileDto
  5. {
  6. [FromQuery]
  7. public bool Cover { get; set; } = false;
  8. [FromQuery]
  9. public string? Provider { get; set; }
  10. [FromForm]
  11. public IFormFileCollection Files { get; set; }
  12. }
  13. }

这里上传参数定义,Cover表示是否覆盖原文件,Provider表示指定那种存储服务。Files则是从Form表单中读取文件流。

FileController

接下来就是把Service包成API对外。

  1. using Microsoft.AspNetCore.Mvc;
  2. using Wheel.Core.Dto;
  3. using Wheel.Services.FileStorageManage;
  4. using Wheel.Services.FileStorageManage.Dtos;
  5. namespace Wheel.Controllers
  6. {
  7. /// <summary>
  8. /// 文件管理
  9. /// </summary>
  10. [Route("api/[controller]")]
  11. [ApiController]
  12. public class FileController : WheelControllerBase
  13. {
  14. private readonly IFileStorageManageAppService _fileStorageManageAppService;
  15. public FileController(IFileStorageManageAppService fileStorageManageAppService)
  16. {
  17. _fileStorageManageAppService = fileStorageManageAppService;
  18. }
  19. /// <summary>
  20. /// 分页查询列表
  21. /// </summary>
  22. /// <param name="request"></param>
  23. /// <returns></returns>
  24. [HttpGet]
  25. public Task<Page<FileStorageDto>> GetFileStoragePageList([FromQuery] FileStoragePageRequest request)
  26. {
  27. return _fileStorageManageAppService.GetFileStoragePageList(request);
  28. }
  29. /// <summary>
  30. /// 上传文件
  31. /// </summary>
  32. /// <param name="uploadFileDto"></param>
  33. /// <returns></returns>
  34. [HttpPost]
  35. public Task<R<List<FileStorageDto>>> UploadFiles(UploadFileDto uploadFileDto)
  36. {
  37. return _fileStorageManageAppService.UploadFiles(uploadFileDto);
  38. }
  39. /// <summary>
  40. /// 下载文件
  41. /// </summary>
  42. /// <param name="id"></param>
  43. /// <returns></returns>
  44. [HttpGet("{id}")]
  45. public async Task<IActionResult> DownloadFile(long id)
  46. {
  47. var result = await _fileStorageManageAppService.DownloadFile(id);
  48. return File(result.Data.Stream, result.Data.ContentType, result.Data.FileName);
  49. }
  50. }
  51. }

DownloadFile返回一个FileResult,浏览器会自动下载。

测试

这里我使用本地的Minio服务进行测试。
查询
image.png
上传
image.png
可以看到我们FileName和Path不一样,默认不覆盖的情况,所有文件在后面自动拼接雪花Id。
下载文件
image.png
这里swagger可以看到有个Download file,点击即可下载出来
image.png
image.png

测试顺利完成,到这我们就完成了我们简单的文件管理功能了。

轮子仓库地址https://github.com/Wheel-Framework/Wheel
欢迎进群催更。

image.png

原文链接:https://www.cnblogs.com/fanshaoO/p/17781914.html

 友情链接:直通硅谷  点职佳  北美留学生论坛

本站QQ群:前端 618073944 | Java 606181507 | Python 626812652 | C/C++ 612253063 | 微信 634508462 | 苹果 692586424 | C#/.net 182808419 | PHP 305140648 | 运维 608723728

W3xue 的所有内容仅供测试,对任何法律问题及风险不承担任何责任。通过使用本站内容随之而来的风险与本站无关。
关于我们  |  意见建议  |  捐助我们  |  报错有奖  |  广告合作、友情链接(目前9元/月)请联系QQ:27243702 沸活量
皖ICP备17017327号-2 皖公网安备34020702000426号