Cleanup search connectors, add some reference -> docs content signposts in various sections (#123733)
This commit is contained in:
Liam Thompson
GitHub
parent
086329c5cb
commit
6b27e420fe
|
|
@ -5,13 +5,11 @@ mapped_pages:
|
|||
|
||||
# Aggregations
|
||||
|
||||
% What needs to be done: Refine
|
||||
:::{note}
|
||||
This section provides detailed **reference information** for aggregations.
|
||||
|
||||
% Scope notes: need to scope the page down to just reference content
|
||||
|
||||
% Use migrated content from existing pages that map to this page:
|
||||
|
||||
% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/search-aggregations.md
|
||||
Refer to [Aggregations](docs-content://explore-analyze/query-filter/aggregations.md) in the **Explore and analyze** section for overview, getting started and conceptual information.
|
||||
:::
|
||||
|
||||
Aggregations are a powerful framework that enables you to perform complex data analysis and summarization over indexed documents. They enable you to extract and compute statistics, trends, and patterns from large datasets.
|
||||
|
||||
|
|
|
|||
|
|
@ -2,6 +2,12 @@
|
|||
|
||||
% TO-DO: Add links to "Data analysis basics"%
|
||||
|
||||
:::{note}
|
||||
This section provides detailed **reference information**.
|
||||
|
||||
Refer to [the text analysis overview](docs-content://manage-data/data-store/text-analysis.md) in the **Manage data** section for overview and conceptual information.
|
||||
:::
|
||||
|
||||
This section contains reference information for text analysis components features, including:
|
||||
|
||||
* [Analyzers](/reference/data-analysis/text-analysis/analyzer-reference.md)
|
||||
|
|
|
|||
|
|
@ -1,5 +1,9 @@
|
|||
---
|
||||
navigation_title: "API tutorial"
|
||||
applies_to:
|
||||
stack: ga
|
||||
serverless:
|
||||
elasticsearch: ga
|
||||
mapped_pages:
|
||||
- https://www.elastic.co/guide/en/elasticsearch/reference/current/es-connectors-tutorial-api.html
|
||||
---
|
||||
|
|
|
|||
|
|
@ -1,4 +1,7 @@
|
|||
---
|
||||
applies_to:
|
||||
stack: ga
|
||||
serverless: ga
|
||||
mapped_pages:
|
||||
- https://www.elastic.co/guide/en/elasticsearch/reference/current/es-connectors-refs.html
|
||||
---
|
||||
|
|
|
|||
|
|
@ -1,13 +1,17 @@
|
|||
---
|
||||
applies_to:
|
||||
stack: ga
|
||||
serverless:
|
||||
elasticsearch: ga
|
||||
mapped_pages:
|
||||
- https://www.elastic.co/guide/en/elasticsearch/reference/current/es-connectors-usage.html
|
||||
---
|
||||
|
||||
# Connectors UI in Kibana [es-connectors-usage]
|
||||
# Connectors UI [es-connectors-usage]
|
||||
|
||||
This document describes operations available to connectors using the UI.
|
||||
|
||||
In the Kibana UI, navigate to **Search > Content > Connectors** from the main menu, or use the [global search field](docs-content://explore-analyze/query-filter/filtering.md#_finding_your_apps_and_objects). Here, you can view a summary of all your connectors and sync jobs, and to create new connectors.
|
||||
In the Kibana or Serverless UI, find Connectors using the [global search field](docs-content://explore-analyze/query-filter/filtering.md#_finding_your_apps_and_objects). Here, you can view a summary of all your connectors and sync jobs, and to create new connectors.
|
||||
|
||||
::::{tip}
|
||||
In 8.12 we introduced a set of [Connector APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-connector) to create and manage Elastic connectors and sync jobs, along with a [CLI tool](https://github.com/elastic/connectors/blob/main/docs/CLI.md). Use these tools if you’d like to work with connectors and sync jobs programmatically, without using the UI.
|
||||
|
|
|
|||
|
|
@ -1,4 +1,6 @@
|
|||
---
|
||||
applies_to:
|
||||
ess: discontinued 9.0.0
|
||||
mapped_pages:
|
||||
- https://www.elastic.co/guide/en/elasticsearch/reference/current/es-native-connectors.html
|
||||
---
|
||||
|
|
|
|||
|
|
@ -1,4 +1,8 @@
|
|||
---
|
||||
applies_to:
|
||||
stack: ga
|
||||
serverless:
|
||||
elasticsearch: ga
|
||||
mapped_pages:
|
||||
- https://www.elastic.co/guide/en/elasticsearch/reference/current/es-connectors-apis.html
|
||||
---
|
||||
|
|
|
|||
|
|
@ -10,24 +10,9 @@ mapped_pages:
|
|||
The following section provides a high-level overview of common architecture approaches for the internal knowledge search use case (AKA workplace search).
|
||||
|
||||
|
||||
## Hybrid architecture [es-connectors-overview-architecture-hybrid]
|
||||
$$$es-connectors-overview-architecture-hybrid$$$
|
||||
|
||||
Data is synced to an Elastic Cloud deployment through managed connectors and/or self-managed connectors. A self-managed search application exposes the relevant data that your end users are authorized to see in a search experience.
|
||||
|
||||
Summary:
|
||||
|
||||
* The best combination in terms of flexibility and out-of-the box functionality
|
||||
* Integrates with Elastic Cloud hosted managed connectors to bring data to Elasticsearch with minimal operational overhead
|
||||
* Self-managed connectors allow enterprises to adhere to strict access policies when using firewalls that don’t allow incoming connections to data sources, while outgoing traffic is easier to control
|
||||
* Provides additional functionality available for self-managed connectors such as the [Extraction Service](/reference/ingestion-tools/search-connectors/es-connectors-content-extraction.md#es-connectors-content-extraction-local)
|
||||
* Basic functionality available for Standard licenses, advanced features for Platinum licenses
|
||||
|
||||
The following diagram provides a high-level overview of the hybrid internal knowledge search architecture.
|
||||
|
||||
:::{image} ../../../images/hybrid-architecture.png
|
||||
:alt: hybrid architecture
|
||||
:class: screenshot
|
||||
:::
|
||||
% ^^ discontinued 9.0.0
|
||||
|
||||
|
||||
## Self-managed architecture [es-connectors-overview-architecture-self-managed]
|
||||
|
|
|
|||
|
|
@ -43,7 +43,7 @@ elasticsearch.api_key: <ELASTICSEARCH_API_KEY>
|
|||
connectors:
|
||||
-
|
||||
connector_id: <CONNECTOR_ID_FROM_KIBANA>
|
||||
service_type: Zoom # sharepoint_online (example)
|
||||
service_type: sharepoint_online # Example value — update this for service type you are connecting to
|
||||
api_key: <CONNECTOR_API_KEY_FROM_KIBANA> # Optional. If not provided, the connector will use the elasticsearch.api_key instead
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,7 @@
|
|||
---
|
||||
applies_to:
|
||||
stack: ga
|
||||
serverless: ga
|
||||
mapped_pages:
|
||||
- https://www.elastic.co/guide/en/elasticsearch/reference/current/es-connectors.html
|
||||
- https://www.elastic.co/guide/en/serverless/current/elasticsearch-ingest-data-through-integrations-connector-client.html
|
||||
|
|
@ -7,18 +10,18 @@ mapped_pages:
|
|||
|
||||
# Search connectors
|
||||
|
||||
% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc):
|
||||
|
||||
$$$es-connectors-native$$$
|
||||
|
||||
|
||||
:::{tip}
|
||||
This page is about Search connectors that synchronize third-party data into Elasticsearch. If you’re looking for Kibana connectors to integrate with services like generative AI model providers, refer to [Kibana Connectors](docs-content://deploy-manage/manage-connectors.md).
|
||||
:::{note}
|
||||
This page is about Search connectors that synchronize third-party data into {{es}}. If you’re looking for Kibana connectors to integrate with services like generative AI model providers, refer to [Kibana Connectors](docs-content://deploy-manage/manage-connectors.md).
|
||||
:::
|
||||
|
||||
A _connector_ is a type of [Elastic integration](https://www.elastic.co/integrations/data-integrations) that syncs data from an original data source to Elasticsearch. Each connector extracts the original files, records, or objects; and transforms them into documents within Elasticsearch.
|
||||
A _connector_ is an Elastic integration that syncs data from an original data source to {{es}}. Use connectors to create searchable, read-only replicas of your data in {{es}}.
|
||||
|
||||
_Connector clients_ are **self-managed** connectors that you run on your own infrastructure. These connectors are written in Python and the source code is available in the [`elastic/connectors`](https://github.com/elastic/connectors/tree/main/connectors/sources) repo.
|
||||
Each connector extracts the original files, records, or objects; and transforms them into documents within {{es}}.
|
||||
|
||||
These connectors are written in Python and the source code is available in the [`elastic/connectors`](https://github.com/elastic/connectors/tree/main/connectors/sources) repo.
|
||||
|
||||
## Available connectors
|
||||
|
||||
|
|
@ -28,9 +31,7 @@ As of Elastic 9.0, managed connectors on Elastic Cloud Hosted are no longer avai
|
|||
::::
|
||||
|
||||
|
||||
Connector clients are available for the following third-party data sources:
|
||||
|
||||
The following connectors are available as self-managed connectors:
|
||||
Connectors are available for the following third-party data sources:
|
||||
|
||||
- [Azure Blob Storage](/reference/ingestion-tools/search-connectors/es-connectors-azure-blob.md)
|
||||
- [Box](/reference/ingestion-tools/search-connectors/es-connectors-box.md)
|
||||
|
|
@ -86,7 +87,6 @@ In order to set up, configure, and run a connector you’ll be moving between yo
|
|||
|
||||
### Data source prerequisites
|
||||
|
||||
|
||||
The first decision you need to make before deploying a connector is which third party service (data source) you want to sync to Elasticsearch. See the list of [available connectors](#available-connectors).
|
||||
The first decision you need to make before deploying a connector is which third party service (data source) you want to sync to {{es}}. See the list of [available connectors](#available-connectors).
|
||||
|
||||
Note that each data source will have specific prerequisites you’ll need to meet to authorize the connector to access its data. For example, certain data sources may require you to create an OAuth application, or create a service account. You’ll need to check the [individual connector documentation](connector-reference.md) for these details.
|
||||
|
|
@ -1,4 +1,7 @@
|
|||
---
|
||||
applies_to:
|
||||
stack: ga
|
||||
serverless: ga
|
||||
mapped_pages:
|
||||
- https://www.elastic.co/guide/en/elasticsearch/reference/current/es-build-connector.html
|
||||
---
|
||||
|
|
@ -10,14 +13,8 @@ Self-managed connectors were initially known as "connector clients". You might f
|
|||
|
||||
::::
|
||||
|
||||
|
||||
Self-managed [Elastic connectors](/reference/ingestion-tools/search-connectors/index.md) are run on your own infrastructure. This means they run outside of your Elastic deployment.
|
||||
|
||||
You can run the [connectors service](#es-connectors-deploy-connector-service) from source or from a Docker container.
|
||||
|
||||
We also have a quickstart option using **Docker Compose**, to spin up all the required services at once: Elasticsearch, Kibana, and the connectors service. Refer to [Docker Compose quickstart](/reference/ingestion-tools/search-connectors/es-connectors-docker-compose-quickstart.md) for more information.
|
||||
|
||||
|
||||
## Availability and Elastic prerequisites [es-build-connector-prerequisites]
|
||||
|
||||
::::{note}
|
||||
|
|
@ -25,7 +22,6 @@ Self-managed connectors currently don’t support Windows. Use this [compatibili
|
|||
|
||||
::::
|
||||
|
||||
|
||||
:::::{dropdown} Expand for Elastic prerequisites information
|
||||
Your Elastic deployment must include the following Elastic services:
|
||||
|
||||
|
|
@ -63,17 +59,16 @@ Note the following information regarding support for self-managed connectors:
|
|||
|
||||
:::::
|
||||
|
||||
## Workflow
|
||||
|
||||
::::{admonition} Data source prerequisites
|
||||
:name: es-build-connector-data-source-prerequisites
|
||||
|
||||
The first decision you need to make before deploying a connector is which third party service (data source) you want to sync to Elasticsearch. Note that each data source will have specific prerequisites you’ll need to meet to authorize the connector to access its data. For example, certain data sources may require you to create an OAuth application, or create a service account.
|
||||
|
||||
You’ll need to check the individual connector documentation for these details.
|
||||
|
||||
::::
|
||||
|
||||
In order to set up, configure, and run a connector you’ll be moving between your third-party service, the Elastic UI, and your terminal. At a high-level, the workflow looks like this:
|
||||
|
||||
1. Satisfy any data source prerequisites (e.g., create an OAuth application).
|
||||
2. Create a connector in the UI (or via the API).
|
||||
3. Deploy the connector service:
|
||||
- [Option 1: Run with Docker](es-connectors-run-from-docker.md) (recommended)
|
||||
- [Option 2: Run from source](es-connectors-run-from-source.md)
|
||||
4. Enter data source configuration details in the UI.
|
||||
|
||||
## Deploy the connector service [es-connectors-deploy-connector-service]
|
||||
|
||||
|
|
@ -87,7 +82,6 @@ You can run the connector service from source or use Docker:
|
|||
* Refer to our [Docker Compose quickstart](/reference/ingestion-tools/search-connectors/es-connectors-docker-compose-quickstart.md) for a quick way to spin up all the required services at once.
|
||||
|
||||
|
||||
|
||||
## Tutorials [es-build-connector-example]
|
||||
|
||||
* Follow our [UI-based tutorial](/reference/ingestion-tools/search-connectors/es-postgresql-connector-client-tutorial.md) to learn how run the self-managed connector service and a set up a self-managed connector, **using the UI**.
|
||||
|
|
@ -95,8 +89,7 @@ You can run the connector service from source or use Docker:
|
|||
|
||||
These examples use the PostgreSQL connector but the basic process is the same for all self-managed connectors.
|
||||
|
||||
|
||||
## Connector testing [es-build-connector-testing]
|
||||
## E2E testing [es-build-connector-testing]
|
||||
|
||||
The connector framework enables you to run end-to-end (E2E) tests on your self-managed connectors, against a real data source.
|
||||
|
||||
|
|
@ -105,7 +98,7 @@ To avoid tampering with a real Elasticsearch instance, E2E tests run an isolated
|
|||
E2E tests use **default** configuration values for the connector. Find instructions about testing in each connector’s documentation.
|
||||
|
||||
|
||||
## Connector framework [es-build-connector-framework]
|
||||
## Build or customize connectors [es-build-connector-framework]
|
||||
|
||||
The Elastic connector framework enables you to:
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,7 @@
|
|||
# Scripting languages
|
||||
|
||||
This section contains the Painless scripting language reference.
|
||||
:::{note}
|
||||
This section provides detailed **reference information** about the the Painless scripting language.
|
||||
|
||||
For an overview of scripting languages in {{es}}, go to [Scripting](docs-content://explore-analyze/scripting.md).
|
||||
Refer to the [scripting languages overview](docs-content://explore-analyze/scripting.md) in the **Explore and analyze** section for an overview of available scripting languages in {{es}}.
|
||||
:::
|
||||
Loading…
Reference in New Issue