There are many different techniques for centralized log management and built-in tools
Sometimes you want to collect all of them to single location, for example AWS S3.

Here is useful script for copying files from local machine to AWS S3 with Ruby SDK v2, it is assumed to run it from EC2 instance which have enabled IAM roles, so no credentials are needed. It will copy log files into different S3 folder names based on instance hostname, which can be handy if you need to download them again:


 #!/usr/local/bin/ruby
 #
 require 'aws-sdk-core'
 def logs()
 #connect to S3Aws.config[:region] = 'eu-west-1'
     s3 = Aws::S3::Client.new
     #get file list with full path
     file_list = Dir.glob("/var/log/httpd/*your_logs_names*")
     #declare time
     vreme = Time.now
     vreme2 = "#{vreme.year}_#{vreme.month}_#{vreme.day}"
     #declare hostname
     hostname = `hostname`.chomp
     #open each log file
     file_list.each {|yo|
         file_open = File.read(yo)
         #obtain only filename, without path:
         file_name = File.basename(yo)
         #upload files:
         s3.put_object(body: file_open, bucket: "bucket-name/#{hostname}/var/log/httpd", key: file_name)
     }
 end

logs()