Awsflow Glue

Awsflow Glue

Manage Glue ETL jobs, triggers, crawlers, databases, tables, workflows, and connections.

When to Use This Skill

Use this skill when the user:

• Asks about Glue ETL jobs, crawlers, or triggers
• Wants to inspect or query the Glue Data Catalog (databases, tables, partitions)
• Needs to start a Glue job run
• Wants to create a Glue job
• Asks about Glue connections or job bookmarks
• Needs to inspect crawl history
• Wants to create or delete databases, tables, crawlers, or workflows
• Needs to update jobs, crawlers, or triggers

Tool: GlueTool

Execute AWS Glue commands including Data Catalog queries and lifecycle operations. ALWAYS provide params object.

Commands

ListJobs

List Glue jobs.

{ "command": "ListJobs", "params": { "MaxResults": 50 } }

Parameters:

GetJob

Get details of a Glue job.

{ "command": "GetJob", "params": { "JobName": "my-etl-job" } }

Parameters:

GetJobRun

Get details of a specific job run.

{ "command": "GetJobRun", "params": { "JobName": "my-etl-job", "RunId": "jr_abc123" } }

Parameters:

GetJobRuns

List all runs of a job.

{ "command": "GetJobRuns", "params": { "JobName": "my-etl-job", "MaxResults": 20 } }

Parameters:

GetJobBookmark

Get the bookmark state for a job.

{ "command": "GetJobBookmark", "params": { "JobName": "my-etl-job" } }

Parameters:

StartJobRun

Start a Glue job run.

{ "command": "StartJobRun", "params": { "JobName": "my-etl-job", "Arguments": { "--input-path": "s3://my-bucket/input/" } } }

Parameters:

CreateJob

Create a new Glue job.

{
  "command": "CreateJob",
  "params": {
    "Name": "my-new-job",
    "Role": "arn:aws:iam::123456789012:role/GlueRole",
    "Command": { "Name": "glueetl", "ScriptLocation": "s3://my-bucket/scripts/etl.py" },
    "WorkerType": "G.1X",
    "NumberOfWorkers": 10
  }
}

Parameters:

ListTriggers

List Glue triggers.

{ "command": "ListTriggers", "params": { "MaxResults": 50 } }

Parameters:

GetTrigger

Get details of a trigger.

{ "command": "GetTrigger", "params": { "Name": "my-trigger" } }

Parameters:

GetTriggers

List triggers with optional filter.

{ "command": "GetTriggers", "params": { "DependencyJobName": "my-job" } }

Parameters:

ListCrawlers

List Glue crawlers.

{ "command": "ListCrawlers", "params": {} }

Parameters:

GetCrawler

Get details of a crawler.

{ "command": "GetCrawler", "params": { "CrawlerName": "my-crawler" } }

Parameters:

GetCrawlers

List crawlers with details.

{ "command": "GetCrawlers", "params": {} }

Parameters:

ListCrawls

List crawl runs for a crawler.

{ "command": "ListCrawls", "params": { "CrawlerName": "my-crawler" } }

Lifecycle Commands

CreateDatabase

Create a new database.

{ "command": "CreateDatabase", "params": { "DatabaseInput": { "Name": "analytics" } } }

CreateTable

Create a new table.

{ "command": "CreateTable", "params": { "DatabaseName": "analytics", "TableInput": { "Name": "events", "StorageDescriptor": { "Columns": [] } } } }

CreateCrawler

Create a crawler.

{ "command": "CreateCrawler", "params": { "Name": "events-crawler", "Role": "arn:aws:iam::123456789012:role/GlueRole", "Targets": { "S3Targets": [{ "Path": "s3://bucket/data/" }] } } }

StartCrawler

Start a crawler.

{ "command": "StartCrawler", "params": { "Name": "events-crawler" } }

UpdateJob

Update an existing job.

{ "command": "UpdateJob", "params": { "JobName": "my-etl-job", "JobUpdate": { "Description": "Updated job" } } }

UpdateCrawler

Update an existing crawler.

{ "command": "UpdateCrawler", "params": { "Name": "events-crawler", "Targets": { "S3Targets": [{ "Path": "s3://bucket/new/" }] } } }

UpdateTrigger

Update a trigger.

{ "command": "UpdateTrigger", "params": { "Name": "daily-trigger", "TriggerUpdate": { "Description": "Updated" } } }

DeleteJob

Delete a job.

{ "command": "DeleteJob", "params": { "JobName": "old-job" } }

DeleteCrawler

Delete a crawler.

{ "command": "DeleteCrawler", "params": { "Name": "events-crawler" } }

DeleteDatabase

Delete a database.

{ "command": "DeleteDatabase", "params": { "Name": "analytics" } }

DeleteTable

Delete a table.

{ "command": "DeleteTable", "params": { "DatabaseName": "analytics", "Name": "events" } }

TagResource

Tag a Glue resource.

{ "command": "TagResource", "params": { "ResourceArn": "arn:aws:glue:...", "Tags": { "env": "prod" } } }

UntagResource

Remove tags from a Glue resource.

{ "command": "UntagResource", "params": { "ResourceArn": "arn:aws:glue:...", "TagKeys": ["env"] } }

Parameters:

GetDatabase

Get a Glue Data Catalog database.

{ "command": "GetDatabase", "params": { "DatabaseName": "my-database" } }

Parameters:

GetDatabases

List all databases in the Data Catalog.

{ "command": "GetDatabases", "params": {} }

Parameters:

GetTable

Get a table definition from the Data Catalog.

{ "command": "GetTable", "params": { "DatabaseName": "my-database", "Name": "my-table" } }

Parameters:

GetTables

List tables in a database.

{ "command": "GetTables", "params": { "DatabaseName": "my-database" } }

Parameters:

GetPartitions

List partitions for a table.

{ "command": "GetPartitions", "params": { "DatabaseName": "my-database", "TableName": "my-table" } }

Parameters:

GetConnections

List or get Glue connections.

{ "command": "GetConnections", "params": { "ConnectionName": "my-connection" } }

Parameters:

GetTags

Get tags for a Glue resource.

{ "command": "GetTags", "params": { "ResourceArn": "arn:aws:glue:..." } }

Parameters:

Related Services

• Glue → CloudWatch Logs: Glue job output logs go to /aws-glue/jobs/output and error logs to /aws-glue/jobs/error. Crawler logs go to /aws-glue/crawlers. Use CloudWatchLogTool to read them
• Glue → S3: Glue jobs read from and write to S3. Crawler targets are often S3 paths. Job scripts are stored in S3. Use S3Tool to inspect
• Glue → IAM: Jobs require IAM roles. Use IAMTool to inspect the execution role
• Glue → Data Catalog → Athena/EMR/Redshift: The Glue Data Catalog is shared with Athena, EMR, and Redshift Spectrum
• Glue → CloudFormation: Glue resources managed by CloudFormation stacks
• Glue → DynamoDB: Glue can read from DynamoDB tables as data sources