{
  "title": "Remove duplicate lines",
  "human_url": "https://happytails.ai/en/text-tools/remove-duplicate-lines/",
  "agent_url": "https://happytails.ai/agents/en/text-tools/remove-duplicate-lines/",
  "language": "en",
  "markdown_url": "https://happytails.ai/en/text-tools/remove-duplicate-lines.md",
  "content": "Remove duplicate lines\nRemove repeated lines while keeping their first occurrence.\n⌑   Runs on your device\nOne copy of each line\nThe first occurrence of each exact line is kept. Line order is preserved, and matching is case-sensitive. Blank lines are treated like any other line.\nExact matches only\n“Apple” and “apple” are different. Clean up extra spaces first if those should not distinguish the lines.\nHow to use Remove duplicate lines\nEnter your text in the input field.\nThe result updates as you type. Adjust the available settings if needed.\nReview the result, then use Copy result or a download link when available.\nWhat to expect\nRemove repeated lines while keeping their first occurrence. Matching is case-sensitive and includes spaces. The original order is preserved; lines that differ only in capitalization remain separate.\nTry a small example\nA\nB\nA\nUseful next steps\nSort lines\nSort lines alphabetically or in natural number order.\nFilter lines\nKeep or remove lines containing a phrase.\nRandomize list\nShuffle lines into a random order.\nAdd line numbers\nNumber each line, starting at 1.",
  "content_format": "text/plain",
  "visibility": "public",
  "links": [
    {
      "name": "Sort lines Sort lines alphabetically or in natural number order.",
      "url": "https://happytails.ai/en/text-tools/sort-lines/"
    },
    {
      "name": "Filter lines Keep or remove lines containing a phrase.",
      "url": "https://happytails.ai/en/text-tools/filter-lines/"
    },
    {
      "name": "Randomize list Shuffle lines into a random order.",
      "url": "https://happytails.ai/en/text-tools/randomize-list/"
    },
    {
      "name": "Add line numbers Number each line, starting at 1.",
      "url": "https://happytails.ai/en/text-tools/add-line-numbers/"
    }
  ],
  "tool": {
    "id": "remove-duplicate-lines",
    "category": "text-tools",
    "built": true,
    "requirements": "Case sensitivity, whitespace normalization and keep-first/last; preserve chosen line order.",
    "implementation": {
      "fields": [],
      "experience": {
        "automatic": true,
        "single": false,
        "label": "Your text",
        "placeholder": "Hello world!",
        "help": "Change the input or settings to update your result. Results update as you type. Up to 1,000,000 characters.",
        "action": null
      },
      "mode": "automatic",
      "interactive": false
    },
    "execution": "POST /api/v1/tools/remove-duplicate-lines/run",
    "api": {
      "id": "remove-duplicate-lines",
      "name": "Remove duplicate lines",
      "category": "text-tools",
      "description": "Remove repeated lines while keeping their first occurrence.",
      "notes": "",
      "human_path": "/en/text-tools/remove-duplicate-lines/",
      "method": "POST",
      "endpoint": "/api/v1/tools/remove-duplicate-lines/run",
      "schema_url": "/api/v1/tools/remove-duplicate-lines",
      "input_schema": {
        "type": "object",
        "additionalProperties": false,
        "properties": {
          "input": {
            "type": "string",
            "maxLength": 1000000,
            "default": "",
            "description": "Plain text input. File tools use files instead unless otherwise documented."
          },
          "options": {
            "type": "object",
            "additionalProperties": false,
            "properties": {}
          }
        }
      },
      "output_schema": {
        "type": "object",
        "required": [
          "tool",
          "result"
        ],
        "properties": {
          "tool": {
            "type": "string"
          },
          "result": {
            "type": "object",
            "required": [
              "text",
              "files"
            ],
            "properties": {
              "text": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "data": {
                "description": "Parsed JSON when the textual result is JSON."
              },
              "name": {
                "type": "string"
              },
              "mime": {
                "type": "string"
              },
              "files": {
                "type": "array",
                "items": {
                  "type": "object",
                  "required": [
                    "name",
                    "mime",
                    "bytes",
                    "base64"
                  ],
                  "properties": {
                    "name": {
                      "type": "string"
                    },
                    "mime": {
                      "type": "string"
                    },
                    "bytes": {
                      "type": "integer"
                    },
                    "base64": {
                      "type": "string",
                      "contentEncoding": "base64"
                    }
                  }
                }
              }
            }
          }
        }
      },
      "processing": "Server; same transformation code as browser tools",
      "limits": {
        "request_bytes": 8388608,
        "input_characters": 1000000,
        "files_bytes": 5242880,
        "max_files": 0,
        "timeout_seconds": 12
      }
    },
    "agent_processing": "Server; same transformation code as browser tools",
    "browser_agent": {
      "supported": true,
      "name": "transform_current_tool",
      "discovery": "WebMCP on the human page in a supporting browser",
      "verification": "See audit/API-AUDIT.md; availability is not verification"
    }
  },
  "markdown": "# Remove duplicate lines\n\n- Human page: https://happytails.ai/en/text-tools/remove-duplicate-lines/\n- Markdown: https://happytails.ai/en/text-tools/remove-duplicate-lines.md\n- Structured JSON: https://happytails.ai/agents/en/text-tools/remove-duplicate-lines/index.json\n- Visibility: public\n\n## Tool capabilities\n\n| Property | Value |\n| --- | --- |\n| Tool ID | `remove-duplicate-lines` |\n| Category | text-tools |\n| Implementation | Registered implementation. Registration alone is not a production-quality guarantee. |\n| Browser execution | Automatic updates after valid input or file selection. |\n| Browser processing | Browser; see tool notes for network behavior. |\n| HTTP API | /api/v1/tools/remove-duplicate-lines/run |\n| Browser-agent interface | transform_current_tool |\n\n### Inputs and settings\n\nText input: Your text.\n\nExample or placeholder shown in the interface (not a validated fixture):\n\n```text\nHello world!\n```\n\nChange the input or settings to update your result. Results update as you type. Up to 1,000,000 characters.\n\nNo additional settings are declared for this tool.\n\n### HTTP contract\n\nAPI calls process supplied data on the server. The website origin is `https://happytails.ai`. Server API hosting is not connected on the public website yet. Use your local server origin to test these requests.\n\n```http\nPOST https://happytails.ai/api/v1/tools/remove-duplicate-lines/run\nContent-Type: application/json\n```\n\n[Detailed operation reference](https://happytails.ai/en/api/tools/remove-duplicate-lines.md) · [Shared API guide](https://happytails.ai/en/api.md)\n\n#### Limits\n\n| Limit | Value |\n| --- | --- |\n| request_bytes | 8388608 |\n| input_characters | 1000000 |\n| files_bytes | 5242880 |\n| max_files | 0 |\n| timeout_seconds | 12 |\n\n#### Complete input schema\n\n```json\n{\n  \"type\": \"object\",\n  \"additionalProperties\": false,\n  \"properties\": {\n    \"input\": {\n      \"type\": \"string\",\n      \"maxLength\": 1000000,\n      \"default\": \"\",\n      \"description\": \"Plain text input. File tools use files instead unless otherwise documented.\"\n    },\n    \"options\": {\n      \"type\": \"object\",\n      \"additionalProperties\": false,\n      \"properties\": {}\n    }\n  }\n}\n```\n\n#### Complete output schema\n\n```json\n{\n  \"type\": \"object\",\n  \"required\": [\n    \"tool\",\n    \"result\"\n  ],\n  \"properties\": {\n    \"tool\": {\n      \"type\": \"string\"\n    },\n    \"result\": {\n      \"type\": \"object\",\n      \"required\": [\n        \"text\",\n        \"files\"\n      ],\n      \"properties\": {\n        \"text\": {\n          \"type\": [\n            \"string\",\n            \"null\"\n          ]\n        },\n        \"data\": {\n          \"description\": \"Parsed JSON when the textual result is JSON.\"\n        },\n        \"name\": {\n          \"type\": \"string\"\n        },\n        \"mime\": {\n          \"type\": \"string\"\n        },\n        \"files\": {\n          \"type\": \"array\",\n          \"items\": {\n            \"type\": \"object\",\n            \"required\": [\n              \"name\",\n              \"mime\",\n              \"bytes\",\n              \"base64\"\n            ],\n            \"properties\": {\n              \"name\": {\n                \"type\": \"string\"\n              },\n              \"mime\": {\n                \"type\": \"string\"\n              },\n              \"bytes\": {\n                \"type\": \"integer\"\n              },\n              \"base64\": {\n                \"type\": \"string\",\n                \"contentEncoding\": \"base64\"\n              }\n            }\n          }\n        }\n      }\n    }\n  }\n}\n```\n\n#### Recorded request and response\n\nSaved fixture from 2026-09-28T18:13:42Z. This is historical evidence, not a new validation or a guarantee for every input.\n\n```json\n{\n  \"input\": \"A\\nB\\nA\",\n  \"options\": {}\n}\n```\n\n```json\n{\n  \"tool\": \"remove-duplicate-lines\",\n  \"result\": {\n    \"text\": \"A\\nB\",\n    \"files\": [],\n    \"name\": \"result.txt\",\n    \"mime\": \"text/plain;charset=utf-8\"\n  }\n}\n```\n\n#### Error and retry handling\n\nFailures return a non-200 status and `error.code` plus `error.message`. Correct invalid input before retrying; retry server-busy responses with bounded backoff. Returned files contain Base64 data, not persistent download URLs. Decode and inspect the output before treating conversion as successful. See the shared API guide for the complete status and timeout rules.\n\n### Browser-agent workflow\n\n1. Open the human page in a browser that supports WebMCP.\n2. Discover the interface exposed by that page; use its actual schema.\n3. Select files on the page first when the tool requires files.\n4. Call `transform_current_tool` with valid input and settings.\n5. Inspect the returned result or error and the displayed output. Retrieve files from the displayed download links.\n\nCalls do not automatically copy or download results. Browser permissions still apply. Interface availability alone is not proof of successful execution.\n\n### Planning specification\n\nCase sensitivity, whitespace normalization and keep-first/last; preserve chosen line order.\n\nThis describes intended requirements. Use the implementation notes, interface schema and observed output to determine current support.\n\n### Complete machine-readable capability record\n\n```json\n{\n  \"id\": \"remove-duplicate-lines\",\n  \"category\": \"text-tools\",\n  \"built\": true,\n  \"requirements\": \"Case sensitivity, whitespace normalization and keep-first/last; preserve chosen line order.\",\n  \"implementation\": {\n    \"fields\": [],\n    \"experience\": {\n      \"automatic\": true,\n      \"single\": false,\n      \"label\": \"Your text\",\n      \"placeholder\": \"Hello world!\",\n      \"help\": \"Change the input or settings to update your result. Results update as you type. Up to 1,000,000 characters.\",\n      \"action\": null\n    },\n    \"mode\": \"automatic\",\n    \"interactive\": false\n  },\n  \"execution\": \"POST /api/v1/tools/remove-duplicate-lines/run\",\n  \"api\": {\n    \"id\": \"remove-duplicate-lines\",\n    \"name\": \"Remove duplicate lines\",\n    \"category\": \"text-tools\",\n    \"description\": \"Remove repeated lines while keeping their first occurrence.\",\n    \"notes\": \"\",\n    \"human_path\": \"/en/text-tools/remove-duplicate-lines/\",\n    \"method\": \"POST\",\n    \"endpoint\": \"/api/v1/tools/remove-duplicate-lines/run\",\n    \"schema_url\": \"/api/v1/tools/remove-duplicate-lines\",\n    \"input_schema\": {\n      \"type\": \"object\",\n      \"additionalProperties\": false,\n      \"properties\": {\n        \"input\": {\n          \"type\": \"string\",\n          \"maxLength\": 1000000,\n          \"default\": \"\",\n          \"description\": \"Plain text input. File tools use files instead unless otherwise documented.\"\n        },\n        \"options\": {\n          \"type\": \"object\",\n          \"additionalProperties\": false,\n          \"properties\": {}\n        }\n      }\n    },\n    \"output_schema\": {\n      \"type\": \"object\",\n      \"required\": [\n        \"tool\",\n        \"result\"\n      ],\n      \"properties\": {\n        \"tool\": {\n          \"type\": \"string\"\n        },\n        \"result\": {\n          \"type\": \"object\",\n          \"required\": [\n            \"text\",\n            \"files\"\n          ],\n          \"properties\": {\n            \"text\": {\n              \"type\": [\n                \"string\",\n                \"null\"\n              ]\n            },\n            \"data\": {\n              \"description\": \"Parsed JSON when the textual result is JSON.\"\n            },\n            \"name\": {\n              \"type\": \"string\"\n            },\n            \"mime\": {\n              \"type\": \"string\"\n            },\n            \"files\": {\n              \"type\": \"array\",\n              \"items\": {\n                \"type\": \"object\",\n                \"required\": [\n                  \"name\",\n                  \"mime\",\n                  \"bytes\",\n                  \"base64\"\n                ],\n                \"properties\": {\n                  \"name\": {\n                    \"type\": \"string\"\n                  },\n                  \"mime\": {\n                    \"type\": \"string\"\n                  },\n                  \"bytes\": {\n                    \"type\": \"integer\"\n                  },\n                  \"base64\": {\n                    \"type\": \"string\",\n                    \"contentEncoding\": \"base64\"\n                  }\n                }\n              }\n            }\n          }\n        }\n      }\n    },\n    \"processing\": \"Server; same transformation code as browser tools\",\n    \"limits\": {\n      \"request_bytes\": 8388608,\n      \"input_characters\": 1000000,\n      \"files_bytes\": 5242880,\n      \"max_files\": 0,\n      \"timeout_seconds\": 12\n    }\n  },\n  \"agent_processing\": \"Server; same transformation code as browser tools\",\n  \"browser_agent\": {\n    \"supported\": true,\n    \"name\": \"transform_current_tool\",\n    \"discovery\": \"WebMCP on the human page in a supporting browser\",\n    \"verification\": \"See audit/API-AUDIT.md; availability is not verification\"\n  }\n}\n```\n\n## Page guide\n\nRemove repeated lines while keeping their first occurrence.\n\n⌑   Runs on your device\n\n## One copy of each line\n\nThe first occurrence of each exact line is kept. Line order is preserved, and matching is case-sensitive. Blank lines are treated like any other line.\n\n## Exact matches only\n\n“Apple” and “apple” are different. Clean up extra spaces first if those should not distinguish the lines.\n\n## How to use Remove duplicate lines\n\n1. Enter your text in the input field.\n2. The result updates as you type. Adjust the available settings if needed.\n3. Review the result, then use Copy result or a download link when available.\n\n## What to expect\n\nRemove repeated lines while keeping their first occurrence. Matching is case-sensitive and includes spaces. The original order is preserved; lines that differ only in capitalization remain separate.\n\n### Try a small example\n\n```\nA\nB\nA\n```\n\n## Useful next steps\n\n- [Sort lines](https://happytails.ai/en/text-tools/sort-lines/): Sort lines alphabetically or in natural number order. ([Markdown](https://happytails.ai/en/text-tools/sort-lines.md))\n\n- [Filter lines](https://happytails.ai/en/text-tools/filter-lines/): Keep or remove lines containing a phrase. ([Markdown](https://happytails.ai/en/text-tools/filter-lines.md))\n\n- [Randomize list](https://happytails.ai/en/text-tools/randomize-list/): Shuffle lines into a random order. ([Markdown](https://happytails.ai/en/text-tools/randomize-list.md))\n\n- [Add line numbers](https://happytails.ai/en/text-tools/add-line-numbers/): Number each line, starting at 1. ([Markdown](https://happytails.ai/en/text-tools/add-line-numbers.md))\n"
}