对于不是很复杂的数据来说，有时使用脚本会更加便捷和方便。它会让我们更快地把数据写入到 Elasticsearch 中去。在今天的文章中，我来展示如何使用 shell 脚本来把一些数据写入到 Elasticsearch 中。
在今天的演示中，我将使用 Elastic Stack 8.2 来进行展示。在 Elasticsearch 的安装中，我们启动 HTTPS 的访问及安全设置。如果你还不是很了解如何安装 Elasticsearch 8.x，请参考我之前的文章 “如何在 Linux，MacOS 及 Windows 上进行安装 Elasticsearch”。
示例一
准备数据
我们首先在一个目录中创建如下的一个文件：
recipes.json
{"index":{"_index": "recipes"}}
{"title":"Egg Salad With Grilled Broccoli and Chili Crisp ","description":"Grilled broccoli is one of life’s simple pleasures and a bona fide crowd favorite. Perfectly grilled broccoli is cooked over high heat to deliver a charred exterior, while the internal flesh retains a crisp-tender bite. The smoky, subtly sweet broccoli feels luxurious served on a bed of creamy egg salad, turning this dish into a satisfying, comforting main meal. Cook your eggs according to your personal preference—I enjoy my yolks slightly soft, but if you prefer them slightly more set, cook the eggs an extra minute. The chili crisp brings additional layers of flavor, offering pops of spice and umami with every bite. Serve with crusty bread, sourdough, or flatbread to mop up every last morsel.","ingredients":["8 large eggs","1½ lb. broccoli (about 3 small heads), cut into florets","2 Tbsp. extra-virgin olive oil","2 tsp. Diamond Crystal or 1 tsp. Morton kosher salt, divided, plus more","1 tsp. freshly ground black pepper, divided, plus more","2 pickles, finely chopped","2 scallions, thinly sliced","½ cup mayonnaise","1 Tbsp. chopped dill","2 tsp. Dijon mustard","1–2 Tbsp. chili crisp (such as Fly by Jing, Lao Gan Ma, or homemade)","1 Tbsp. toasted sesame seeds","Country-style bread, sourdough, or flatbread (for serving)"],"instructions":["Bring a medium pot of water to a boil. Carefully lower eggs into water with a slotted spoon and cook 8 minutes for medium-cooked or 9 minutes for hard-cooked. Transfer eggs with slotted spoon to a large bowl of ice water and let cool. Prepare a grill for high heat. (Or, heat a grill pan over high.) Place broccoli in a large bowl, drizzle with oil, and sprinkle 1 tsp. Diamond Crystal or ½ tsp. Morton kosher salt and ½ tsp. freshly ground black pepper over; toss to coat. Grill, undisturbed, until grill marks appear, about 2 minutes. Turn and cook on opposite side until charred in spots and broccoli is just tender, about 2 minutes. Peel eggs and coarsely chop; transfer to a medium bowl. Add pickles, scallions, mayonnaise, dill, mustard, 1 tsp. Diamond Crystal or ½ tsp. Morton kosher salt, and ½ tsp. freshly ground black pepper and mix well. Taste and season with more salt and pepper if needed. To serve, spoon egg salad on a platter and arrange broccoli on top. Drizzle with chili crisp as desired, then sprinkle with sesame seeds. Serve with bread.","Bring a medium pot of water to a boil. Carefully lower eggs into water with a slotted spoon and cook 8 minutes for medium-cooked or 9 minutes for hard-cooked. Transfer eggs with slotted spoon to a large bowl of ice water and let cool.","Prepare a grill for high heat. (Or, heat a grill pan over high.) Place broccoli in a large bowl, drizzle with oil, and sprinkle 1 tsp. Diamond Crystal or ½ tsp. Morton kosher salt and ½ tsp. freshly ground black pepper over; toss to coat. Grill, undisturbed, until grill marks appear, about 2 minutes. Turn and cook on opposite side until charred in spots and broccoli is just tender, about 2 minutes.","Peel eggs and coarsely chop; transfer to a medium bowl. Add pickles, scallions, mayonnaise, dill, mustard, 1 tsp. Diamond Crystal or ½ tsp. Morton kosher salt, and ½ tsp. freshly ground black pepper and mix well. Taste and season with more salt and pepper if needed.","To serve, spoon egg salad on a platter and arrange broccoli on top. Drizzle with chili crisp as desired, then sprinkle with sesame seeds. Serve with bread."],"picture_link":"https://assets.epicurious.com/photos/60ff0fc093749898de0e59d7/6:4/w_274%2Ch_169/GrilledBroccoliEggSalad_RECIPE_072221_19251.jpg","author":"Hetty McKinnon","rating":{"ratingValue":"0","bestRating":"4","worstRating":"0","prepare_again":"0"},"date":"July 2021","summary":{"yield":"4 Servings","active-time":"22 minutes","total-time":"22 minutes"},"url":"http://www.epicurious.com/recipes/food/views/egg-salad-grilled-broccoli-and-chili-crisp"}
{"index":{"_index": "recipes"}}
{"title":"Chili Crisp With Oats ","description":"This chile–oat crisp can be used like chile oil or as a stand-alone sauce. Drizzling it on noodles instantly makes it a meal. At Asian grocers, you can find a product called Lao Gan Ma Spicy Chili Crisp, hailing from China’s Guizhou province. This garlicky, spicy, crunchy condiment is served with everything from noodles to ice cream to peanut brittle. The defining characteristic of this chile sauce is the crispness, and in my version I’ve included old-fashioned rolled oats and coconut flakes for extra crunch. You will find me adding it to pasta, eggs, jook, pizza and, of course, roasted vegetables. Vary the amount of spice in your chili crisp according to your tastes.","ingredients":["3 shallots, finely diced","2 garlic cloves, finely chopped","1-inch (2.5 cm) piece of ginger, peeled and finely chopped","1 cinnamon stick","1 cup (100 g) old-fashioned rolled oats","½ cup (30 g) coconut flakes, roughly chopped","3 tablespoons toasted white sesame seeds","3 tablespoons red chile flakes","1½ cups (375 ml) vegetable or other neutral oil","2 tablespoons toasted sesame oil","About 1 tablespoon sea salt"],"instructions":["To make the chili crisp, place the shallots, garlic, ginger, cinnamon stick, oats, coconut flakes, sesame seeds, chile flakes and oils in a saucepan over medium heat. Bring to a gentle simmer, swirling the pan every now and then, then reduce the heat to medium–low and cook for 20–25 minutes until everything is crispy. Strain the oil through a sieve over a bowl and leave the oat mixture to cool in the sieve—this will allow it to crisp up further. Set the oil aside. Once the crispy oat mixture is completely cool, stir it back into the oil and season with the sea salt. Leave the cinnamon stick in the oil, as it will continue to flavor it. Store the chili crisp in a sterilized jar. It will keep well for several months.","To make the chili crisp, place the shallots, garlic, ginger, cinnamon stick, oats, coconut flakes, sesame seeds, chile flakes and oils in a saucepan over medium heat. Bring to a gentle simmer, swirling the pan every now and then, then reduce the heat to medium–low and cook for 20–25 minutes until everything is crispy.","Strain the oil through a sieve over a bowl and leave the oat mixture to cool in the sieve—this will allow it to crisp up further. Set the oil aside. Once the crispy oat mixture is completely cool, stir it back into the oil and season with the sea salt. Leave the cinnamon stick in the oil, as it will continue to flavor it. Store the chili crisp in a sterilized jar. It will keep well for several months."],"picture_link":"https://assets.epicurious.com/photos/6107f447d9a60ab80c34106c/6:4/w_274%2Ch_169/TAWL_SALAD_RootVegChilliOatCrisp-1.jpg","author":"Hetty McKinnon","rating":{"ratingValue":"4","bestRating":"4","worstRating":"0","prepare_again":"100"},"date":"April 2021","summary":null,"url":"http://www.epicurious.com/recipes/food/views/homemade-chile-crisp-oats-hetty-mckinnon"}
{"index":{"_index": "recipes"}}
{"title":"Esqueixada de Montaña (Cured Trout With Tomato, Black Olive, and Onion) ","description":"Esqueixar means “to shred” and that’s what’s done to salt cod in this traditional Catalonian cold dish. I’m using trout instead, which is as prevalent in Asheville as it is in Spain. For the restaurant, we get fresh trout delivered the day it is caught and serve it raw. Since I’m not sure everyone can get trout that fresh, I lightly cure it here to retain its silky tenderness. (Note that the trout is cured, not cooked, if you’re concerned about consuming raw fish.) Since the fillets aren’t as flaky as cod, they don’t get shredded. In thin slices, though, they capture the spirit of esqueixada.","ingredients":["1 lemon","1½ teaspoon honey","½ teaspoon kosher salt","¼ cup extra-virgin olive oil","2 large super-ripe tomatoes (10 ounces)","2 garlic cloves, minced","1 large rosemary sprig","1 teaspoon kosher salt","½ cup extra-virgin olive oil","1 pound trout fillets, pin bones removed","¼ cup gin","8 juniper berries, crushed with the flat side of a knife","1 teaspoon black peppercorns, crushed","½ cup kosher salt","½ cup granulated sugar","3 tablespoons Lemon Vinaigrette","¼ cup thinly sliced sweet onion","¼ cup diced pitted Kalamata olives","1 lemon"],"instructions":["To make the lemon vinaigrette: To make the vinaigrette, zest the lemon into the bowl of a food processor. Trim the top and bottom of the lemon, then cut off the peel and pith. Holding the lemon flesh over the food processor bowl, cut out the segments by slicing between the membranes. Squeeze any juice remaining in the membrane, then squeeze any juice in the peel and pith. Add the honey and salt to the bowl and process. With the machine running, add the oil in a slow, steady stream until emulsified. The vinaigrette can be refrigerated in an airtight container for up to 3 days. Bring to room temperature and shake well before using.","To make the vinaigrette, zest the lemon into the bowl of a food processor. Trim the top and bottom of the lemon, then cut off the peel and pith. Holding the lemon flesh over the food processor bowl, cut out the segments by slicing between the membranes. Squeeze any juice remaining in the membrane, then squeeze any juice in the peel and pith. Add the honey and salt to the bowl and process. With the machine running, add the oil in a slow, steady stream until emulsified. The vinaigrette can be refrigerated in an airtight container for up to 3 days. Bring to room temperature and shake well before using.","To make the grated fresh tomatoes: Cut the tomatoes in half through their equators. Set a box grater over a large bowl. Grate the cut sides of the tomatoes on the large holes until only the skin remains; discard the skins. Stir in the garlic. Cut the rosemary sprig in half, then bruise the sprigs by gently pounding with the dull edge of a knife and add to the tomato mixture. Whisk in the salt and oil. Cover tightly with plastic wrap and refrigerate overnight for the flavors to meld.","Cut the tomatoes in half through their equators.","Set a box grater over a large bowl. Grate the cut sides of the tomatoes on the large holes until only the skin remains; discard the skins. Stir in the garlic. Cut the rosemary sprig in half, then bruise the sprigs by gently pounding with the dull edge of a knife and add to the tomato mixture. Whisk in the salt and oil. Cover tightly with plastic wrap and refrigerate overnight for the flavors to meld.","To make the cured trout: Find a container or shallow dish just large enough to hold the fillets snugly in a single layer. Line the container with plastic wrap with overhang on all sides. Place the fillets in the plastic wrap, skin side down. Pour the gin on top. Mix the juniper, pepper- corns, salt, and sugar in a small bowl, then spread over the fillets to evenly coat. Cover with the plastic wrap overhang and then cover the whole container with plastic wrap. Refrigerate for 4 hours. Unwrap the trout, rinse off the cure under cold water, and pat dry. Cut the trout off the skin; discard the skin. Cut the fillets into 1/8-inch-thick slices. Remove the rosemary from the tomatoes and spread the tomatoes on a serving platter. Top with the trout slices. Drizzle the lemon vinaigrette on top, then sprinkle with the onion and olives. Zest the lemon directly on top and serve immediately.","Find a container or shallow dish just large enough to hold the fillets snugly in a single layer. Line the container with plastic wrap with overhang on all sides. Place the fillets in the plastic wrap, skin side down.","Pour the gin on top. Mix the juniper, pepper- corns, salt, and sugar in a small bowl, then spread over the fillets to evenly coat. Cover with the plastic wrap overhang and then cover the whole container with plastic wrap. Refrigerate for 4 hours.","Unwrap the trout, rinse off the cure under cold water, and pat dry. Cut the trout off the skin; discard the skin. Cut the fillets into 1/8-inch-thick slices. Remove the rosemary from the tomatoes and spread the tomatoes on a serving platter.","Top with the trout slices. Drizzle the lemon vinaigrette on top, then sprinkle with the onion and olives. Zest the lemon directly on top and serve immediately."],"picture_link":"https://assets.epicurious.com/photos/60fecf88d2ec312335e99405/6:4/w_274%2Ch_169/CuredTrout_sharpening_V1.jpg","author":"Katie Button","rating":{"ratingValue":"0","bestRating":"4","worstRating":"0","prepare_again":"0"},"date":"October 2016","summary":{"yield":"Serves 4 as a main dish or 8 as a small plate"},"url":"http://www.epicurious.com/recipes/food/views/cured-trout-with-tomato-black-olive-and-onion"}
它的文件格式如上所示。请注意在最后一行是一个空行。
写入数据
我们接下来创建一个如下的 sample-index.json 文件。这个文件是用来定义一个索引的 mapping 的：
sample-index.json
{
"aliases": {},
"mappings": {
"properties": {
"author": {
"type": "keyword",
"eager_global_ordinals": true,
"fields": {
"text": {
"type": "text"
}
}
},
"date": {
"type": "date",
"format": "[MMMM yyyy]",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"description": {
"type": "text",
"analyzer": "english"
},
"ingredients": {
"type": "text",
"analyzer": "english"
},
"instructions": {
"type": "text",
"analyzer": "english"
},
"picture_link": {
"type": "keyword"
},
"rating": {
"properties": {
"bestRating": {
"type": "float"
},
"prepare_again": {
"type": "float"
},
"ratingValue": {
"type": "float"
},
"worstRating": {
"type": "float"
}
}
},
"recipe_id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"summary": {
"properties": {
"active-time": {
"type": "keyword"
},
"total-time": {
"type": "keyword"
},
"yield": {
"type": "keyword"
}
}
},
"title": {
"type": "text",
"analyzer": "english",
"fields": {
"keyword": {
"type": "keyword"
},
"suggestion": {
"type": "completion",
"analyzer": "english",
"preserve_separators": false,
"preserve_position_increments": false,
"max_input_length": 50
}
}
},
"url": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}如果我们不定义这个 mapping，那么写入的数据会被 Elasticsearch 自动识别，并按照第一个写入的文档的数据来进行猜测每个字段的类型。我们首先定义一个 mapping，这样可以避免字段被错误地映射，同时也可以按照我们的意图来进行映射以节省存储成本等。
接下来，我们创建一个如下的一个脚本：
load.sh
#!/bin/bash
printf "\n**Load.sh loads an index for recipes into Elasticsearch**\n\n"
echo -n "Elasticsearch cluster URL: "
read url
echo -n "Username: "
read username
echo -n "Password: "
read -s password
echo
echo -n "certificate: "
read -s cert
echo
# load index template
if curl -f -XPUT "$url/recipes" --cacert $cert -u $username:$password -H 'Content-Type: application/json' -d "@sample-index.json"
then echo " Created index for recipes"
else echo " Could not load index"
exit
fi
if curl -f -XPOST "$url/_bulk" --cacert $cert -u $username:$password -H 'Content-Type: application/x-ndjson' --data-binary @recipes.json
then echo "Write documents successfully"
else echo "Failed to write documents"
fi
printf "\n*Loaded index successfully\n"
在这里，我们需要输入 Elasticsearch 的访问地址，用户名，密码以及访问 Elasticsearch 所需要的证书地址：
$ pwd
/Users/liuxg/data/script_load
$ ls
load.sh recipes.json sample-index.json这样我们的文件如上所示。接下来，我们需要把 load.sh 修改为可执行文件：
chmod a+x load.sh我们接下来按照如下的命令来进行运行：
./load.sh
如上所示，我们已经成功地把数据写入到 Elasticsearch 中去了。我们可以使用 Kibana 来进行检查：
我们可以查看一下它的 mapping：
示例二
在上面的方法中，我们可以看到每个文档的前面有一个这样的语句：
{"index":{"_index": "recipes"}}在实际的应用中，这样的格式是不多见的，除非我们自己预处理。我们大多简单的 JSON 格式的文档是这样的：
recipes1.json
{"title":"Egg Salad With Grilled Broccoli and Chili Crisp ","description":"Grilled broccoli is one of life’s simple pleasures and a bona fide crowd favorite. Perfectly grilled broccoli is cooked over high heat to deliver a charred exterior, while the internal flesh retains a crisp-tender bite. The smoky, subtly sweet broccoli feels luxurious served on a bed of creamy egg salad, turning this dish into a satisfying, comforting main meal. Cook your eggs according to your personal preference—I enjoy my yolks slightly soft, but if you prefer them slightly more set, cook the eggs an extra minute. The chili crisp brings additional layers of flavor, offering pops of spice and umami with every bite. Serve with crusty bread, sourdough, or flatbread to mop up every last morsel.","ingredients":["8 large eggs","1½ lb. broccoli (about 3 small heads), cut into florets","2 Tbsp. extra-virgin olive oil","2 tsp. Diamond Crystal or 1 tsp. Morton kosher salt, divided, plus more","1 tsp. freshly ground black pepper, divided, plus more","2 pickles, finely chopped","2 scallions, thinly sliced","½ cup mayonnaise","1 Tbsp. chopped dill","2 tsp. Dijon mustard","1–2 Tbsp. chili crisp (such as Fly by Jing, Lao Gan Ma, or homemade)","1 Tbsp. toasted sesame seeds","Country-style bread, sourdough, or flatbread (for serving)"],"instructions":["Bring a medium pot of water to a boil. Carefully lower eggs into water with a slotted spoon and cook 8 minutes for medium-cooked or 9 minutes for hard-cooked. Transfer eggs with slotted spoon to a large bowl of ice water and let cool. Prepare a grill for high heat. (Or, heat a grill pan over high.) Place broccoli in a large bowl, drizzle with oil, and sprinkle 1 tsp. Diamond Crystal or ½ tsp. Morton kosher salt and ½ tsp. freshly ground black pepper over; toss to coat. Grill, undisturbed, until grill marks appear, about 2 minutes. Turn and cook on opposite side until charred in spots and broccoli is just tender, about 2 minutes. Peel eggs and coarsely chop; transfer to a medium bowl. Add pickles, scallions, mayonnaise, dill, mustard, 1 tsp. Diamond Crystal or ½ tsp. Morton kosher salt, and ½ tsp. freshly ground black pepper and mix well. Taste and season with more salt and pepper if needed. To serve, spoon egg salad on a platter and arrange broccoli on top. Drizzle with chili crisp as desired, then sprinkle with sesame seeds. Serve with bread.","Bring a medium pot of water to a boil. Carefully lower eggs into water with a slotted spoon and cook 8 minutes for medium-cooked or 9 minutes for hard-cooked. Transfer eggs with slotted spoon to a large bowl of ice water and let cool.","Prepare a grill for high heat. (Or, heat a grill pan over high.) Place broccoli in a large bowl, drizzle with oil, and sprinkle 1 tsp. Diamond Crystal or ½ tsp. Morton kosher salt and ½ tsp. freshly ground black pepper over; toss to coat. Grill, undisturbed, until grill marks appear, about 2 minutes. Turn and cook on opposite side until charred in spots and broccoli is just tender, about 2 minutes.","Peel eggs and coarsely chop; transfer to a medium bowl. Add pickles, scallions, mayonnaise, dill, mustard, 1 tsp. Diamond Crystal or ½ tsp. Morton kosher salt, and ½ tsp. freshly ground black pepper and mix well. Taste and season with more salt and pepper if needed.","To serve, spoon egg salad on a platter and arrange broccoli on top. Drizzle with chili crisp as desired, then sprinkle with sesame seeds. Serve with bread."],"picture_link":"https://assets.epicurious.com/photos/60ff0fc093749898de0e59d7/6:4/w_274%2Ch_169/GrilledBroccoliEggSalad_RECIPE_072221_19251.jpg","author":"Hetty McKinnon","rating":{"ratingValue":"0","bestRating":"4","worstRating":"0","prepare_again":"0"},"date":"July 2021","summary":{"yield":"4 Servings","active-time":"22 minutes","total-time":"22 minutes"},"url":"http://www.epicurious.com/recipes/food/views/egg-salad-grilled-broccoli-and-chili-crisp"}
{"title":"Chili Crisp With Oats ","description":"This chile–oat crisp can be used like chile oil or as a stand-alone sauce. Drizzling it on noodles instantly makes it a meal. At Asian grocers, you can find a product called Lao Gan Ma Spicy Chili Crisp, hailing from China’s Guizhou province. This garlicky, spicy, crunchy condiment is served with everything from noodles to ice cream to peanut brittle. The defining characteristic of this chile sauce is the crispness, and in my version I’ve included old-fashioned rolled oats and coconut flakes for extra crunch. You will find me adding it to pasta, eggs, jook, pizza and, of course, roasted vegetables. Vary the amount of spice in your chili crisp according to your tastes.","ingredients":["3 shallots, finely diced","2 garlic cloves, finely chopped","1-inch (2.5 cm) piece of ginger, peeled and finely chopped","1 cinnamon stick","1 cup (100 g) old-fashioned rolled oats","½ cup (30 g) coconut flakes, roughly chopped","3 tablespoons toasted white sesame seeds","3 tablespoons red chile flakes","1½ cups (375 ml) vegetable or other neutral oil","2 tablespoons toasted sesame oil","About 1 tablespoon sea salt"],"instructions":["To make the chili crisp, place the shallots, garlic, ginger, cinnamon stick, oats, coconut flakes, sesame seeds, chile flakes and oils in a saucepan over medium heat. Bring to a gentle simmer, swirling the pan every now and then, then reduce the heat to medium–low and cook for 20–25 minutes until everything is crispy. Strain the oil through a sieve over a bowl and leave the oat mixture to cool in the sieve—this will allow it to crisp up further. Set the oil aside. Once the crispy oat mixture is completely cool, stir it back into the oil and season with the sea salt. Leave the cinnamon stick in the oil, as it will continue to flavor it. Store the chili crisp in a sterilized jar. It will keep well for several months.","To make the chili crisp, place the shallots, garlic, ginger, cinnamon stick, oats, coconut flakes, sesame seeds, chile flakes and oils in a saucepan over medium heat. Bring to a gentle simmer, swirling the pan every now and then, then reduce the heat to medium–low and cook for 20–25 minutes until everything is crispy.","Strain the oil through a sieve over a bowl and leave the oat mixture to cool in the sieve—this will allow it to crisp up further. Set the oil aside. Once the crispy oat mixture is completely cool, stir it back into the oil and season with the sea salt. Leave the cinnamon stick in the oil, as it will continue to flavor it. Store the chili crisp in a sterilized jar. It will keep well for several months."],"picture_link":"https://assets.epicurious.com/photos/6107f447d9a60ab80c34106c/6:4/w_274%2Ch_169/TAWL_SALAD_RootVegChilliOatCrisp-1.jpg","author":"Hetty McKinnon","rating":{"ratingValue":"4","bestRating":"4","worstRating":"0","prepare_again":"100"},"date":"April 2021","summary":null,"url":"http://www.epicurious.com/recipes/food/views/homemade-chile-crisp-oats-hetty-mckinnon"}
{"title":"Esqueixada de Montaña (Cured Trout With Tomato, Black Olive, and Onion) ","description":"Esqueixar means “to shred” and that’s what’s done to salt cod in this traditional Catalonian cold dish. I’m using trout instead, which is as prevalent in Asheville as it is in Spain. For the restaurant, we get fresh trout delivered the day it is caught and serve it raw. Since I’m not sure everyone can get trout that fresh, I lightly cure it here to retain its silky tenderness. (Note that the trout is cured, not cooked, if you’re concerned about consuming raw fish.) Since the fillets aren’t as flaky as cod, they don’t get shredded. In thin slices, though, they capture the spirit of esqueixada.","ingredients":["1 lemon","1½ teaspoon honey","½ teaspoon kosher salt","¼ cup extra-virgin olive oil","2 large super-ripe tomatoes (10 ounces)","2 garlic cloves, minced","1 large rosemary sprig","1 teaspoon kosher salt","½ cup extra-virgin olive oil","1 pound trout fillets, pin bones removed","¼ cup gin","8 juniper berries, crushed with the flat side of a knife","1 teaspoon black peppercorns, crushed","½ cup kosher salt","½ cup granulated sugar","3 tablespoons Lemon Vinaigrette","¼ cup thinly sliced sweet onion","¼ cup diced pitted Kalamata olives","1 lemon"],"instructions":["To make the lemon vinaigrette: To make the vinaigrette, zest the lemon into the bowl of a food processor. Trim the top and bottom of the lemon, then cut off the peel and pith. Holding the lemon flesh over the food processor bowl, cut out the segments by slicing between the membranes. Squeeze any juice remaining in the membrane, then squeeze any juice in the peel and pith. Add the honey and salt to the bowl and process. With the machine running, add the oil in a slow, steady stream until emulsified. The vinaigrette can be refrigerated in an airtight container for up to 3 days. Bring to room temperature and shake well before using.","To make the vinaigrette, zest the lemon into the bowl of a food processor. Trim the top and bottom of the lemon, then cut off the peel and pith. Holding the lemon flesh over the food processor bowl, cut out the segments by slicing between the membranes. Squeeze any juice remaining in the membrane, then squeeze any juice in the peel and pith. Add the honey and salt to the bowl and process. With the machine running, add the oil in a slow, steady stream until emulsified. The vinaigrette can be refrigerated in an airtight container for up to 3 days. Bring to room temperature and shake well before using.","To make the grated fresh tomatoes: Cut the tomatoes in half through their equators. Set a box grater over a large bowl. Grate the cut sides of the tomatoes on the large holes until only the skin remains; discard the skins. Stir in the garlic. Cut the rosemary sprig in half, then bruise the sprigs by gently pounding with the dull edge of a knife and add to the tomato mixture. Whisk in the salt and oil. Cover tightly with plastic wrap and refrigerate overnight for the flavors to meld.","Cut the tomatoes in half through their equators.","Set a box grater over a large bowl. Grate the cut sides of the tomatoes on the large holes until only the skin remains; discard the skins. Stir in the garlic. Cut the rosemary sprig in half, then bruise the sprigs by gently pounding with the dull edge of a knife and add to the tomato mixture. Whisk in the salt and oil. Cover tightly with plastic wrap and refrigerate overnight for the flavors to meld.","To make the cured trout: Find a container or shallow dish just large enough to hold the fillets snugly in a single layer. Line the container with plastic wrap with overhang on all sides. Place the fillets in the plastic wrap, skin side down. Pour the gin on top. Mix the juniper, pepper- corns, salt, and sugar in a small bowl, then spread over the fillets to evenly coat. Cover with the plastic wrap overhang and then cover the whole container with plastic wrap. Refrigerate for 4 hours. Unwrap the trout, rinse off the cure under cold water, and pat dry. Cut the trout off the skin; discard the skin. Cut the fillets into 1/8-inch-thick slices. Remove the rosemary from the tomatoes and spread the tomatoes on a serving platter. Top with the trout slices. Drizzle the lemon vinaigrette on top, then sprinkle with the onion and olives. Zest the lemon directly on top and serve immediately.","Find a container or shallow dish just large enough to hold the fillets snugly in a single layer. Line the container with plastic wrap with overhang on all sides. Place the fillets in the plastic wrap, skin side down.","Pour the gin on top. Mix the juniper, pepper- corns, salt, and sugar in a small bowl, then spread over the fillets to evenly coat. Cover with the plastic wrap overhang and then cover the whole container with plastic wrap. Refrigerate for 4 hours.","Unwrap the trout, rinse off the cure under cold water, and pat dry. Cut the trout off the skin; discard the skin. Cut the fillets into 1/8-inch-thick slices. Remove the rosemary from the tomatoes and spread the tomatoes on a serving platter.","Top with the trout slices. Drizzle the lemon vinaigrette on top, then sprinkle with the onion and olives. Zest the lemon directly on top and serve immediately."],"picture_link":"https://assets.epicurious.com/photos/60fecf88d2ec312335e99405/6:4/w_274%2Ch_169/CuredTrout_sharpening_V1.jpg","author":"Katie Button","rating":{"ratingValue":"0","bestRating":"4","worstRating":"0","prepare_again":"0"},"date":"October 2016","summary":{"yield":"Serves 4 as a main dish or 8 as a small plate"},"url":"http://www.epicurious.com/recipes/food/views/cured-trout-with-tomato-black-olive-and-onion"}
请注意最后的一行是空行。那么我改如何操作呢？
我们需要使用脚本把文档的每一行读出来，并在它的前面添加如下的一行：
{"create": {}}它表示创建一个文档。
为了能够正确地让索引的数据具有正确的 mapping 及配置，我们创建如下的一个 template.json 文件：
template.json
{
"order": 0,
"index_patterns": ["recipes*"],
"mappings": {
"properties": {
"author": {
"type": "keyword",
"eager_global_ordinals": true,
"fields": {
"text": {
"type": "text"
}
}
},
"date": {
"type": "date",
"format": "[MMMM yyyy]",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"description": {
"type": "text",
"analyzer": "english"
},
"ingredients": {
"type": "text",
"analyzer": "english"
},
"instructions": {
"type": "text",
"analyzer": "english"
},
"picture_link": {
"type": "keyword"
},
"rating": {
"properties": {
"bestRating": {
"type": "float"
},
"prepare_again": {
"type": "float"
},
"ratingValue": {
"type": "float"
},
"worstRating": {
"type": "float"
}
}
},
"recipe_id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"summary": {
"properties": {
"active-time": {
"type": "keyword"
},
"total-time": {
"type": "keyword"
},
"yield": {
"type": "keyword"
}
}
},
"title": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
},
"suggestion": {
"type": "completion",
"analyzer": "english",
"preserve_separators": false,
"preserve_position_increments": false,
"max_input_length": 50
}
},
"analyzer": "english"
},
"url": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}基于这种设计，我们创建如下的脚本：
load_json.sh
#!/bin/bash
input="recipes1.json"
output="output.log"
counter=0
max_rows=2
create='{"create": {}}'
bulk_data=$'\n'
es_url="https://localhost:9200" # Elasticsearch endpoint
username="elastic"
password="sJdGUSQdTY3OrD2E7uKC" # elastic account password
cert="/Users/liuxg/elastic/elasticsearch-8.2.0/config/certs/http_ca.crt"
echo "Indexing a template ..."
curl -XPUT --cacert $cert -u $username:$password "$es_url/_template/recipes" -H 'Content-Type: application/json' --data-binary @template.json > "$output"
echo "Reading blogs from $input..."
while read -r doc
do
let "counter=counter+1"
bulk_data+="$create"$'\n'"$doc"$'\n'
if [ $counter -eq $max_rows ]
then
echo "Indexing $counter documents..."
bulk_data+=$'\n'
echo "$bulk_data" | tee temp.json > /dev/null
if curl -f -XPOST "$es_url/recipes/_bulk" --cacert $cert -u $username:$password -H 'Content-Type: application/json' --data-binary @temp.json
then echo "Write documents successfully"
else echo "Failed to write documents"
fi
rm -rf temp.json
counter=0
bulk_data=$'\n'
fi
done < "$input"
if [ $counter -lt $max_rows ] && [ $counter -gt 0 ]
then
echo "Indexing $counter documents..."
bulk_data+=$'\n'
echo "$bulk_data" | tee temp.json > /dev/null
if curl -f -XPOST "$es_url/recipes/_bulk" --cacert $cert -u $username:$password -H 'Content-Type: application/json' --data-binary @temp.json
then echo "Write documents successfully"
else echo "Failed to write documents"
fi
rm -rf temp.json
fi我们使用如下的命令把上面的脚本变为可执行的文件：
chmod a+x load_json.sh我们使用如下的命令来执行这个脚本：
./load_json.sh
我们看到文件已经被成功地写入了。我们可以在 Kibana 中进行查看：
GET recipes/_count{
"count" : 3,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
}
}上面显示有3个文档已经被成功地写入。
我们可以查看它的 mapping：
GET recipes/_mapping{
"recipes" : {
"mappings" : {
"properties" : {
"author" : {
"type" : "keyword",
"eager_global_ordinals" : true,
"fields" : {
"text" : {
"type" : "text"
}
}
},
"date" : {
"type" : "date",
"format" : "[MMMM yyyy]",
"fields" : {
"keyword" : {
"type" : "keyword"
}
}
},
"description" : {
"type" : "text",
"analyzer" : "english"
},
"ingredients" : {
"type" : "text",
"analyzer" : "english"
},
"instructions" : {
"type" : "text",
"analyzer" : "english"
},
"picture_link" : {
"type" : "keyword"
},
"rating" : {
"properties" : {
"bestRating" : {
"type" : "float"
},
"prepare_again" : {
"type" : "float"
},
"ratingValue" : {
"type" : "float"
},
"worstRating" : {
"type" : "float"
}
}
},
"recipe_id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"summary" : {
"properties" : {
"active-time" : {
"type" : "keyword"
},
"total-time" : {
"type" : "keyword"
},
"yield" : {
"type" : "keyword"
}
}
},
"title" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword"
},
"suggestion" : {
"type" : "completion",
"analyzer" : "english",
"preserve_separators" : false,
"preserve_position_increments" : false,
"max_input_length" : 50
}
},
"analyzer" : "english"
},
"url" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
}从上面，我们可以看出来文档的 mapping 是正确的。
今天的文章elasticsearch命令行操作_linux写入内容到文件「建议收藏」分享到此就结束了，感谢您的阅读。
版权声明：本文内容由互联网用户自发贡献，该文观点仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 举报，一经查实，本站将立刻删除。
如需转载请保留出处：https://bianchenghao.cn/68504.html
