Home > Casper, Jamf Pro, Tech > Pre-Loading AWS S3 Buckets With Jamf Pro

Pre-Loading AWS S3 Buckets With Jamf Pro

November 4, 2017

It’s been far too long and way too much has been going on. I took a new position back in February with our corporate office with the lofty goal of migrating 15,000 endpoints into a centralized Jamf Pro instance. We worked with Jamf and with our internal resources to determine where to place the infrastructure and finally settled on AWS. I’ll go into more detail in other posts because for now, I wanted to talk about sideloading packages in S3.

Most of us have dev or test environments. Ours is in AWS alongside our production environment. With the release of Jamf Pro 10 this past week, it meant it was time to upgrade our dev instances to 10. Not only do we host the servers in AWS, but we also host the distribution point for our dev instance on S3. As you may be aware, each time you create a new cloud distribution point with Jamf Pro a new S3 bucket is spun up. That’s fine, but what if you have a bunch of test packages already in a dev S3 bucket that you want to use? That’s where I found myself this evening, and I figured out how to move those into the new S3 bucket that Jamf Pro created.

After spinning up your new Jamf Pro servers and creating your S3 bucket in Jamf Pro, head over to the AWS Console to manage your S3 buckets. Locate your original bucket and select all of the files in the bucket.

Next go under the More button and choose Copy.

Now go find the new bucket that was created by Jamf Pro and go under the More button in the bucket and choose Paste.

After getting the files over, we now have to tell Jamf Pro that the packages are there. Using the AWS CLI from your computer, grab a listing of your bucket:

aws s3 ls s3://<yourbucketaddress> >> ~/mypackages.txt
view raw AWS CLI LS hosted with ❤ by GitHub

Now that you’ve got a list of all packages in that S3 bucket, you’ll want to get rid of all of the extra data so that you only have the package filename. I used Excel to delete columns and combine columns if there were spaces (use the Open menu in Excel and choose “Delimited” and use Spaces as your delimiter). Once you have a clean list, save that out of Excel and then open it in your favorite text editor. TextMate is my go to for this. You’ll want to save the text out as a CSV with LF only.

We now have a clean list that we can send through a handy API script to stub out the packages in Jamf Pro. We will use the following script to import this data into the Jamf Pro server. Be sure to edit the JSS address in the script before running.

# Name: addPackages.sh
# Date: 27 Mar 2018
# Author: Steve Wood (steve.wood@omnicomgroup.com)
# Purpose: used to stub out packages in the Jamf Pro database
# The CSV file needs to be saved as a UNIX file with LF, not CR
# Version: 1.1
# Update: 27 Mar 2018 - fixing an error in the counter
# A good portion of this script is re-purposed from the script posted in the following JAMF Nation article:
# https://jamfnation.jamfsoftware.com/discussion.html?id=13118#respond
#Verify we can read the file
data=`cat $file`
if [[ "$data" == "" ]]; then
echo "Unable to read the file path specified"
echo "Ensure there are no spaces and that the path is correct"
exit 1
totalqty=`awk -F, 'END {printf "%s\n", NR}' $file`
#Set a counter for the loop
echo $data
#Loop through the CSV and submit data to the API
while [ $counter -lt $totalqty ]
line=`echo "$data" | head -n $counter | tail -n 1`
package=`echo "$line" | awk -F , '{print $1}'`
output=`curl -sS -k -i -u ${jssAPIUsername}:${jssAPIPassword} -X POST -H "Content-Type: text/xml" -d "<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?>$apiData" ${jssAddress}/JSSResource/packages`
echo $output
error=`echo $output | grep "Conflict"`
if [[ $error != "" ]]; then
echo "The following packages could not be created:"
printf -- '%s\n' "${duplicates[@]}"
exit 0
view raw addPackages.sh hosted with ❤ by GitHub

Let’s open up terminal and run our script, feeding it our cleaned up text file.

/path/to/script/addPackages.sh <jssuser> <jsspass> /path/to/textfile.csv

That’s it. As that script runs it will add the packages to the Jamf Pro server so you don’t have to. This will help speed up the process of creating new dev instances each time.

In some of my next posts I’ll discuss how we are using the Server app on machines as distribution points, and utilizing simple scripts with LaunchDaemons to keep them in sync.

Categories: Casper, Jamf Pro, Tech
%d bloggers like this: