搭建文件服务器FastDFS

资料包:
百度云盘链接: https://pan.baidu.com/s/1dmMJhMCkfBY_rJlpnw9o-w 密码: r5pc

1. 安装gcc、openssl、pcre、zlib等依赖包。

资料包中已经有了相关安装包
可以参考https://blog.csdn.net/weixin_57105109/article/details/126707004

#检查是否安装了gcc
gcc -v
#安装gcc
yum -y install gcc
#安装openssl
tar -zxvf openssl-3.0.5.tar.gz
cd openssl-3.0.5/
./config
make && make install

tar -zxvf pcre2-10.40.tar.gz
cd pcre2-10.40/
./config
make && make install

tar -zxvf zlib-1.2.12.tar.gz
cd zlib-1.2.12/
./config
make && make install

2. 安装libfastcommon

获取libfastcommon安装包:
wget https://github.com/happyfish100/libfastcommon/archive/V1.0.38.tar.gz
解压安装包:tar -zxvf V1.0.38.tar.gz
进入目录:cd libfastcommon-1.0.38
执行编译:./make.sh
安装:./make.sh install

3. 安装FastDFS

获取fdfs安装包
wget https://github.com/happyfish100/fastdfs/archive/V5.11.tar.gz
 
解压安装包:tar -zxvf V5.11.tar.gz
进入目录:cd fastdfs-5.11
执行编译:./make.sh
安装:./make.sh install

配置Trancker

进入/etc/fdfs目录,修改tracker.conf.sample为tracker.conf
 
编辑tracker.conf:vim tracker.conf,修改相关参数
 
base_path=/home/fastdfs/tracker
 
#tracker文件夹里的data文件夹和logs文件夹,会自动生成,否则必须提前创建好 

http.server_port=80 #http端口,保持不变,尽量与nginx相同
 
启动tracker(支持start|stop|restart):
 
/usr/bin/fdfs_trackerd  /etc/fdfs/tracker.conf  start

配置Storage

#进入/etc/fdfs目录,修改storage.conf.sample为storage.conf
cp storage.conf.sample storage.conf
 
编辑storage.conf:vi storage.conf,修改相关参数:
 
base_path=/home/fastdfs/storage
#storage文件夹里的data文件夹和logs文件夹,会自动生成,否则必须提前创建好 。
store_path0=/home/fastdfs/storage  #如果为空,则使用base_path 
#配置该storage监听的tracker的ip和port
tracker_server=10.122.149.211:22122 # 这里不能使用127.0.0.1 。22122端口也尽量不用改
 
#启动storage
/usr/bin/fdfs_storaged /etc/fdfs/storage.conf start

4. 安装nginx和fastdfs-nginx-module模块

安装nginx

下载Nginx安装包
 
wget https://nginx.org/download/nginx-1.18.0.tar.gz
 
下载fastdfs-nginx-module安装包
 
wget https://github.com/happyfish100/fastdfs-nginx-module/archive/V1.20.tar.gz
 
解压nginx:tar -zxvf nginx-1.15.2.tar.gz
 
解压fastdfs-nginx-module:tar -xvf V1.20.tar.gz
 
进入nginx目录:cd nginx-1.15.2
 
安装依赖的库
 
apt-get update yum install libpcre3 libpcre3-dev openssl libssl-dev libperl-dev pcre pcre-devel lib zlib-devel
 
修改fastdfs-nginx-module/src/config文件,修改一下内容
ngx_module_incs="/usr/include/fastdfs /usr/include/fastcommon/"
 
CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/include/fastcommon/"
 
#配置,并加载fastdfs-nginx-module模块:
#注意这里用自己的路径,不一定是:/root/fastdfs-nginx-module-1.20/src/
./configure --prefix=/usr/local/nginx --add-module=/root/fastdfs-nginx-module-1.20/src/
 
编译安装:
make && make install

nginx常用命令
cd /usr/local/nginx/sbin
./nginx   #启动
./nginx -t #检查配置文件 
./nginx -s reload #重启

配置nginx和fdfs

1. 配置mod-fastdfs.conf,并拷贝到/etc/fdfs文件目录下
cd fastdfs-nginx-module-1.20/src/
cp mod_fastdfs.conf /etc/fdfs
 
2. 进入/etc/fdfs修改mod-fastdfs.conf:
base_path=/home/fastdfs 
tracker_server=10.122.149.211:22122 #tracker的地址
url_have_group_name=true #url是否包含group名称 
store_path0=/home/fastdfs/storage #文件存储的位置
 
3. 配置nginx,80端口server增加location 启动nginx
cd /usr/local/nginx/conf/ 
vi nginx.conf
 
location ~/M00{
    root /home/fastdfs/storage/data;
    ngx_fastdfs_module;
}
 
4. 最后需要拷贝fastdfs解压目录中的http.conf和mime.types:
cd /usr/local/src/fastdfs-5.11/conf 
cp mime.types http.conf /etc/fdfs/

5. FastDFS常用命令测试

#上传文件
1. 进入/etc/fdfs目录,有cp命令拷贝client.conf.sample,为client.conf;
 
2. 修改client.conf相关配置:
base_path=/home/fastdfs/tracker  //tracker服务器文件路径
    tracker_server=10.122.149.211:22122 //tracker服务器IP地址和端口号
    http.tracker_server_port=80 
 
3. 新建一个测试文档1.txt,内容为abc
4. 命令:
/usr/bin/fdfs_upload_file  <config_file> <local_filename>
 
5. 示例:
/usr/bin/fdfs_upload_file  /etc/fdfs/client.conf 1.txt
 
6. 查看结果,进入storage的data目录:
/home/fastdfs/storage/data/00/00/CnqV01trmeyAbAN0AAAABLh3frE677.txt
 
7. 通过wget和浏览器方式访问成功:这里访问的是配置的nginx服务器,注意nginx的端口号
wget http://10.122.149.211/group1/M00/00/00/CnqV01trmeyAbAN0AAAABLh3frE677.txt
 
#下载文件:
1. 命令:
/usr/bin/fdfs_download_file <config_file> <file_id> [local_filename]
2. 示例:
/usr/bin/fdfs_download_file /etc/fdfs/client.conf 
group1/M00/00/00/CnqV01trmeyAbAN0AAAABLh3frE677.txt a.txt
 
#删除文件:
1.命令:
/usr/bin/fdfs_delete_file <config_file> <file_id>
 
2.示例:
/usr/bin/fdfs_delete_file /etc/fdfs/client.conf
group1/M00/00/00/CnqV01trmeyAbAN0AAAABLh3frE677.txt

6.FastDFS使用

添加依赖,或者直接使用jar包

<dependency>
    <groupId>com.github.tobato</groupId>
    <artifactId>fastdfs-client</artifactId>
    <version>1.26.7</version>
</dependency>

FastDFS配置。application. yml文件

fdfs:
  # 文件下载拉取地址。nginx的ip和端口号
  web-server-url: https://img.xxx.com
  so-timeout: 3000
  connect-timeout: 1200
  thumb-image: # 缩略图
    width: 60
    height: 60
  tracker-list: 192.168.2.9:22122
  pool:
    jmx-enabled: false #禁止JMX重复注册,影响springboot JMX监控

FdfsUtil工具类

private static final List<String> IMAGE_TYPE = 
            Arrays.asList("jpg", "jpeg", "png", "bmp", "tif", "gif", "webp");
    private static final List<String> VIDEO_TYPE = Arrays.asList("mp4", "avi");
    private static final String IMG_LINUX_LOCATION = "/usr/local/data/img";

    /**
     * 前端 文件上传
     * @param fastFileStorageClient
     * @param file
     * @param imageServer
     * @param needThumbnail
     * @param needSize
     * @return
     * @throws IOException
     */
    public static UploadFile upload(FastFileStorageClient fastFileStorageClient, 
                                    MultipartFile file, String imageServer, 
                                    Byte needThumbnail, Byte needSize) 
                                    throws IOException {
        UploadFile uploadFile = new UploadFile(
                  file.getOriginalFilename(), file.getSize(), file.getContentType());

        uploadFile(fastFileStorageClient, file.getInputStream(), 
                   uploadFile, imageServer, needThumbnail, needSize);
        return uploadFile;
    }

    /**
     * 后端 文件上传
     * @param fastFileStorageClient
     * @param file
     * @param imageServer
     * @param needThumbnail
     * @param needSize
     * @return
     */
    public static UploadFile upload(FastFileStorageClient fastFileStorageClient, 
                                    File file, String imageServer, 
                                    Byte needThumbnail, Byte needSize) {
        try (InputStream is = new FileInputStream(file)) {
            UploadFile uploadFile = new UploadFile(
                       file.getName(), file.length(), null);

            uploadFile(fastFileStorageClient, is, uploadFile, 
                       imageServer, needThumbnail, needSize);

            return uploadFile;
        } catch (Exception e){
            throw new CommonException("上传文件失败", e);
        }
    }

    /**
     * 上传文件,处理图片
     * @param fastFileStorageClient
     * @param inputStream
     * @param uploadFile
     * @param imageServer
     * @param needThumbnail
     * @param needSize
     * @throws IOException
     */
    private static void uploadFile(FastFileStorageClient fastFileStorageClient, 
                                   InputStream inputStream, UploadFile uploadFile, 
                                   String imageServer, Byte needThumbnail, 
                                   Byte needSize) throws IOException {
        log.info("文件上传 originalFileName={}, fileSize={}", 
                 uploadFile.getOriginalFilename(), uploadFile.getFileSize());

        // 获取文件后缀
        String fileExtName = FilenameUtils.getExtension(
                             uploadFile.getOriginalFilename()).toLowerCase();

        // 限制图片文件上传大小
        if(isSupportType(fileExtName)){
            validateUploadImage(uploadFile, fileExtName);
        }

        // 缓存文件流
        byte[] fileStream = IOUtils.toByteArray(inputStream);

        // 上传原图或文件
        uploadOrigFile(fastFileStorageClient, uploadFile, 
                      imageServer, fileStream, fileExtName);

        // 上传图片缩略图
        if(isSupportType(fileExtName)){
            uploadImageThumbnails(fastFileStorageClient, uploadFile, 
                  imageServer, fileStream, fileExtName, needThumbnail, needSize);
        }

        log.info("上传文件地址={}", uploadFile.getFullFilename());
    }

    /**
     * 限制图片文件上传大小
     * @param uploadFile
     * @param fileExtName
     */
    private static void validateUploadImage(UploadFile uploadFile, 
                                            String fileExtName){
        // gif文件,大小校验,超过4兆,不上传
        if("gif".equals(fileExtName)){
            if(uploadFile.getFileSize() > 4 * 1024 * 1024){
                throw new CommonException("GIF动图大小不能超过4M");
            }
        } else if(VIDEO_TYPE.contains(fileExtName)){
            if(uploadFile.getFileSize() > 40 * 1024 * 1024){
                throw new CommonException("视频大小不能超过40M");
            }
        } else{
            // 普通图片文件不能超过10M
            if(uploadFile.getFileSize() > 10 * 1024 * 1024){
                throw new CommonException("上传单张图片不能超过10M");
            }
        }
    }

    /**
     * 上传原文件
     * @param fastFileStorageClient
     * @param uploadFile
     * @param imageServer
     * @param fileStream
     * @param fileExtName
     */
    private static void uploadOrigFile(FastFileStorageClient fastFileStorageClient, 
                                       UploadFile uploadFile, String imageServer,
                                       byte[] fileStream, String fileExtName){
        try (ByteArrayInputStream is = new ByteArrayInputStream(fileStream)) {
            StorePath originalStorePath = fastFileStorageClient.uploadFile(
                      is, uploadFile.getFileSize(), fileExtName, null);
            uploadFile.setFullFilename(imageServer.concat("/").concat(
                       originalStorePath.getFullPath()));
        } catch (IOException e) {
            log.error("upload Image error", e.getCause());
            throw new FdfsUploadImageException("upload Image error", e.getCause());
        }
    }

    /**
     * 上传图片缩略图
     * @param fastFileStorageClient
     * @param uploadFile
     * @param imageServer
     * @param fileStream
     * @param fileExtName
     * @param needThumbnail
     * @param needSize
     * @throws IOException
     */
    private static void uploadImageThumbnails(
                                        FastFileStorageClient fastFileStorageClient, 
                                        UploadFile uploadFile, String imageServer,
                                        byte[] fileStream, String fileExtName, 
                                        Byte needThumbnail, Byte needSize) 
                                        throws IOException {
        // 默认图片缩略图取原图
        uploadFile.setThumbnail(uploadFile.getFullFilename());
        // 缩略图也需要计算图片尺寸大小
        if(Constant.YES.equals(needSize) || Constant.YES.equals(needThumbnail)){
            // 获取图片尺寸
            BufferedImage bufferedImage = ImageIO.read(new ByteArrayInputStream(
                                          fileStream));

            if(bufferedImage != null){
                uploadFile.setWidth(bufferedImage.getWidth());
                uploadFile.setHeight(bufferedImage.getHeight());

                // 压缩处理
                if(Constant.YES.equals(needThumbnail)){
                    InputStream is = null;
                    try {
                        ByteArrayOutputStream out = new ByteArrayOutputStream();
                        float outputQuality = getOutputQuality(
                                               uploadFile.getFileSize());
                        if(!"jpg".equals(fileExtName)){
                            // 先转成jpg
                            String newPicPath = IMG_LINUX_LOCATION + File.separator 
                                             + UUID.randomUUID().toString() + ".jpg";
                            Thumbnails.of(new ByteArrayInputStream(fileStream))
                                                  .scale(1f).toFile(newPicPath);
                            // 原比例不变,大小压缩
                            Thumbnails.of(newPicPath).scale(1f)
                                  .outputQuality(outputQuality).toOutputStream(out);
                            // 删除jpg图片
                            new File(newPicPath).delete();
                        } else {
                            Thumbnails.of(new ByteArrayInputStream(fileStream))
                                        .scale(1f).outputQuality(outputQuality)
                                        .toOutputStream(out);
                        }
                        is = new ByteArrayInputStream(out.toByteArray());

                        // 上传缩略图
                        StorePath thumbnailStorePath = fastFileStorageClient
                          .uploadFile(is, is.available(), fileExtName, null);
                        uploadFile.setThumbnail(imageServer.concat("/")
                            .concat(thumbnailStorePath.getFullPath()));

                        log.info("上传缩略图地址={}", uploadFile.getThumbnail());
                    } catch (IOException e) {
                        log.error("upload ThumbImage error", e.getCause());
                        throw new FdfsUploadImageException(
                                  "upload ThumbImage error",e.getCause());
                    } finally {
                        IOUtils.closeQuietly(is);
                    }
                }
            }
        }
    }

    /**
     * 设置压缩比例
     * @param fileSize
     * @return
     */
    private static float getOutputQuality(long fileSize){
        if (fileSize < 64 * 1024) {
            return 1f;
        } else if (fileSize < 256 * 1024) {
            return 0.4f;
        } else if (fileSize < 2 * 1024 * 1024) {
            return 0.2f;
        }

        return 0.1f;
    }

    /**
     * 判断文件格式是否支持
     * @param fileExtName
     * @return
     */
    private static boolean isSupportType(String fileExtName) {
        return IMAGE_TYPE.contains(fileExtName) || VIDEO_TYPE.contains(fileExtName);
    }

    /**
     * 删除文件
     * @param fileUrl 文件访问地址
     * @param fastFileStorageClient
     */
    public static void deleteFile(FastFileStorageClient fastFileStorageClient, 
                                                               String fileUrl) {
        if (StringUtils.isEmpty(fileUrl)) {
            return;
        }
        try {
            StorePath storePath = StorePath.parseFromUrl(fileUrl);
            fastFileStorageClient.deleteFile(
                                storePath.getGroup(), storePath.getPath());
        } catch (FdfsUnsupportStorePathException e) {
            log.warn("异常信息:", e);
        }
    }

}

UploadFile

@Data
public class UploadFile{
  private String originalFilename;
  private long fileSize;
  private String contentType;
  private String fullFilename;
  public UploadFile(){}
 public UploadFile(String originalFilename,long fileSize,String contentType){
    this.originalFilename=originalFilename;
    this.fileSize=fileSize;
    this.contentType=contentType;
  }
}

FdfsController

@Slf4j
@RestController
@RequestMapping("/fdfs")
@Api(tags = "文件上传")
public class FdfsController {

    @Value("${fdfs.web-server-url}")
    private String imageServer;

    @Autowired
    private FastFileStorageClient fastFileStorageClient;

    @PostMapping(value = "/upload", consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
                                    produces = MediaType.APPLICATION_JSON_VALUE)
    @ApiOperation(value = "文件上传", httpMethod = "POST")
    public String upload(@RequestParam("file") MultipartFile file,
     @RequestParam(value = "needSize", defaultValue = "0") Byte needSize,
     @RequestParam(value = "needThumbnail", defaultValue = "0") Byte needThumbnail) {
        try {
            long start = System.currentTimeMillis();

            // 获取文件后缀
            String fileExtName = FilenameUtils.getExtension(
                                  file.getOriginalFilename()).toUpperCase();

            UploadFile uploadFile = FdfsUtil.upload(
                 fastFileStorageClient, file, imageServer, needThumbnail, needSize);

            long end = System.currentTimeMillis();

            log.info("文件上传耗时:{} 耗秒", (end - start));

             return JSON.toJSONString(uploadFile);
        } catch (Exception e) {
            log.error("上传文件异常:", e);
        }

        log.warn(String.format("文件 %s 上传失败", file.getOriginalFilename()));

        return "上传文件异常";
    }

}

==========服务端上传文件================
  @Value("${fdfs.web-server-url}")
  private String imageServer;

  @Autowired
  private FastFileStorageClient fastFileStorageClient;
  File file = new File("xxxx");
  UploadFile uploadFile = FdfsUtil.upload(
                    fastFileStorageClient, file, imageServer, null, null);
  String fileUrl = uploadFile.getFullFilename();
  file.deleteOnExit();
  return fileUrl;
最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
  • 序言:七十年代末,一起剥皮案震惊了整个滨河市,随后出现的几起案子,更是在滨河造成了极大的恐慌,老刑警刘岩,带你破解...
    沈念sama阅读 217,657评论 6 505
  • 序言:滨河连续发生了三起死亡事件,死亡现场离奇诡异,居然都是意外死亡,警方通过查阅死者的电脑和手机,发现死者居然都...
    沈念sama阅读 92,889评论 3 394
  • 文/潘晓璐 我一进店门,熙熙楼的掌柜王于贵愁眉苦脸地迎上来,“玉大人,你说我怎么就摊上这事。” “怎么了?”我有些...
    开封第一讲书人阅读 164,057评论 0 354
  • 文/不坏的土叔 我叫张陵,是天一观的道长。 经常有香客问我,道长,这世上最难降的妖魔是什么? 我笑而不...
    开封第一讲书人阅读 58,509评论 1 293
  • 正文 为了忘掉前任,我火速办了婚礼,结果婚礼上,老公的妹妹穿的比我还像新娘。我一直安慰自己,他们只是感情好,可当我...
    茶点故事阅读 67,562评论 6 392
  • 文/花漫 我一把揭开白布。 她就那样静静地躺着,像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上,一...
    开封第一讲书人阅读 51,443评论 1 302
  • 那天,我揣着相机与录音,去河边找鬼。 笑死,一个胖子当着我的面吹牛,可吹牛的内容都是我干的。 我是一名探鬼主播,决...
    沈念sama阅读 40,251评论 3 418
  • 文/苍兰香墨 我猛地睁开眼,长吁一口气:“原来是场噩梦啊……” “哼!你这毒妇竟也来了?” 一声冷哼从身侧响起,我...
    开封第一讲书人阅读 39,129评论 0 276
  • 序言:老挝万荣一对情侣失踪,失踪者是张志新(化名)和其女友刘颖,没想到半个月后,有当地人在树林里发现了一具尸体,经...
    沈念sama阅读 45,561评论 1 314
  • 正文 独居荒郊野岭守林人离奇死亡,尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 37,779评论 3 335
  • 正文 我和宋清朗相恋三年,在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 39,902评论 1 348
  • 序言:一个原本活蹦乱跳的男人离奇死亡,死状恐怖,灵堂内的尸体忽然破棺而出,到底是诈尸还是另有隐情,我是刑警宁泽,带...
    沈念sama阅读 35,621评论 5 345
  • 正文 年R本政府宣布,位于F岛的核电站,受9级特大地震影响,放射性物质发生泄漏。R本人自食恶果不足惜,却给世界环境...
    茶点故事阅读 41,220评论 3 328
  • 文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹,春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 31,838评论 0 22
  • 文/苍兰香墨 我抬头看了看天上的太阳。三九已至,却和暖如春,着一层夹袄步出监牢的瞬间,已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 32,971评论 1 269
  • 我被黑心中介骗来泰国打工, 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留,地道东北人。 一个月前我还...
    沈念sama阅读 48,025评论 2 370
  • 正文 我出身青楼,却偏偏与公主长得像,于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子,可洞房花烛夜当晚...
    茶点故事阅读 44,843评论 2 354

推荐阅读更多精彩内容