The Haystack elves have found in Santa’s basement an ancient lathe machine that would be perfect to craft wooden toys. Too bad the instructions are written in a language they can’t really understand:
b'\x84\xa8metadata\x80\xb1max_loops_allowedd\xaacomponents\x86\xa9converter\x82\xa4type\xd92haystack.components.converters.html.HTMLToDocument\xafinit_parameters\x80\xa7fetcher\x82\xa4type\xd9<haystack.components.fetchers.link_content.LinkContentFetcher\xafinit_parameters\x84\xb0raise_on_failure\xc3\xabuser_agents\x91\xd9#haystack/LinkContentFetcher/0.152.0\xaeretry_attempts\x02\xa7timeout\x03\xa3llm\x82\xa4type\xd92haystack.components.generators.openai.GPTGenerator\xafinit_parameters\x85\xaamodel_name\xadgpt-3.5-turbo\xb2streaming_callback\xc0\xacapi_base_url\xb9https://api.openai.com/v1\xb1generation_kwargs\x80\xadsystem_prompt\xc0\xaeprompt_builder\x82\xa4type\xd99haystack.components.builders.prompt_builder.PromptBuilder\xafinit_parameters\x81\xa8template\xd9\x8a Acc ding to these docu nts:\n{% for oc in documents %} {{ doc.con }} {% endfor %}\nAnswer the given qu tion: {{question}} Answer: \xa6ranker\x82\xa4type\xd9Phaystack.components.rankers.transformers_similarity.TransformersSimilarityRanker\xafinit_parameters\x84\xa6device\xa3cpu\xb2model_name_or_path\xd9$cross-encoder/ms-marco-MiniLM-L-6-v2\xa5token\xc0\xa5top_k\x03\xa8splitter\x82\xa4type\xd9Dhaystack.components.preprocessors.document_splitter.DocumentSplitter\xafinit_parameters\x83\xa8split_by\xa4word\xacsplit_length2\xadsplit_overlap\x00\xabconnections\x95\x82\xa6sender\xb3converter.documents\xa8receiver\xb2splitter.documents\x82\xa6sender\xaffetcher.streams\xa8receiver\xb1converter.sources\x82\xa6sender\xb5prompt_builder.prompt\xa8receiver\xaallm.prompt\x82\xa6sender\xb0ranker.documents\xa8receiver\xb8prompt_builder.documents\x82\xa6sender\xb2splitter.documents\xa8receiver\xb0ranker.documents'
Elf Massi seems to have an intuition, he saw the same characters in a painting in his grandparents' house! “That’s MessagePack!” he claims, “it’s like JSON but harder, better, faster, stronger and it looks like it contains a pipeline, we can translate it!”.
If you wan to help the elves, your challenge is to write a custom marshaller you can use to load the MessagePack pipeline and save it again in YAML format so that they can understand it and put in use the ancient lathe machine.
Beware the instructions are pretty old, and part of the text might have faded away… Once you get the instructions in YAML format, try to see if you can figure out the missing parts!
When the YAML pipeline is ready you can use this data to finally get the instructions about how to use the lathe:
{
"prompt_builder": {"question": "how do I start a lathe?"},
"ranker": {"query": "how do I start a lathe?"},
"fetcher": {"urls": ["https://en.wikipedia.org/wiki/Lathe"]}
}
⏳ Hint: A custom marshaller isn’t much different from the YAML marshaller that already comes with Haystack.
📚 Useful documentation:
💚 Here is the Starter Colab