Remove Duplicates (duplicate)

The duplicate filter is designed to remove duplicate points based on their short name (traditionally a waypoint's name on the GPS receiver), and/or their location (to a precision of 6 decimals). This filter supports two options that specify how duplicates will be recognized, shortname and location. At least one of these options is required.

Example 4.17. Using the duplicate filter to suppress points with the same name and location

This command line removes points that have duplicate short names and duplicate locations. The result would be a gpx file that more than likely contains only unique points and point data.

gpsbabel -i gpx -f 1.gpx -f 2.gpx -x duplicate,location,shortname -o gpx -F merged_with_no_dupes.gpx

shortname option

Suppress duplicate waypoints based on name.

This option is the one most often used with the duplicate filter. This option instructs the duplicate filter to remove any waypoints that share a short name with a waypoint that has come before. This option might be used to remove duplicates if you are merging two datasets that were each created in part from a common ancestor dataset.

location option

Suppress duplicate waypoint based on coords.

This option causes the duplicate filter to remove any additional waypoint that has the same coordinates (to six decimal degrees) as a waypoint that came before. This option may be used to remove duplicate waypoints if the names are not expected to be the same. It also might be used along with the shortname option to remove duplicate waypoints if the names of several unrelated groups of waypoints might be the same.

all option

Suppress all instances of duplicates.

When this option is specified, GPSBabel will remove all instances of a duplicated waypoint, not just the second and subsequent instances. If your input file contains waypoints A, B, B, and C, the output file will contain waypoints A, B, and C without the all option, or just A and C with the all option.

Example 4.18. Using the duplicate filter to implement an "ignore list."

This option may be used to implement an "ignore list." In the following example, the duplicate filter is used to remove a list of waypoints to be ignored from a larger collection of waypoints:

gpsbabel -i gpx -f waypoints.gpx -i csv -f to_ignore.csv -x duplicate,shortname,all -o gpx -F filtered.gpx

correct option

Use coords from duplicate points.

This option is used to change the locations of waypoints without losing any of the other associated information. When this option is specified, the latitude and longitude from later duplicates will replace the latitude and longitude in the original waypoint.

As an example, this option may be used to adjust the locations of "puzzle" geocaches in a Groundspeak pocket query:

Example 4.19. Using the duplicate filter to correct the locations of "puzzle" geocaches

gpsbabel -i gpx -f 43622.gpx -i csv -f corrections.csv -x duplicate,shortname,correct -o gpx -F 43622-corrected.gpx

After this command is run, the waypoints in the output file will have all of the descriptive information from 43622.gpx, but waypoints that were also found in corrections.csv will have their coordinates replaced with the coordinates from that file.