Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revamped Elasticsearch example #13025

Merged
1 commit merged into from
Sep 4, 2015
Merged

Conversation

pires
Copy link
Contributor

@pires pires commented Aug 21, 2015

/cc @satnam6502

You can find the sources for building the images here.

@k8s-bot
Copy link

k8s-bot commented Aug 21, 2015

Can one of the admins verify that this patch is reasonable to test? (reply "ok to test", or if you trust the user, reply "add to whitelist")

If this message is too spammy, please complain to ixdy.

@pires pires force-pushed the example_elasticsearch branch from 83cc0ac to c1ef927 Compare August 21, 2015 11:53
@pires
Copy link
Contributor Author

pires commented Aug 21, 2015

I can see now that am missing the LoadBalancer for the service. @satnam6502 need help with this one.

@satnam6502
Copy link
Contributor

@pires thank you. I will try this out for myself and get back to you -- including the use of LoadBalancer.

@k8s-github-robot k8s-github-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Aug 27, 2015
@k8s-github-robot
Copy link

Labelling this PR as size/XXL

@pires
Copy link
Contributor Author

pires commented Aug 28, 2015

@satnam6502 I imagine this week has been crazy but did you get a chance to try this out?

@satnam6502
Copy link
Contributor

Indeed, let me see what I can do today (my last day).

@satnam6502 satnam6502 removed their assignment Aug 28, 2015
@satnam6502
Copy link
Contributor

I am very sorry -- I have totally run out of time on my last day and I won't have a chance to review this PR. Please can it be re-assigned to someone else? I vote for @a-robinson or @mikedanese

@a-robinson
Copy link
Contributor

@jimmidyson, would you be up for reviewing this?

@jimmidyson
Copy link
Member

Only if I can be very harsh on @pires ;)

@pires
Copy link
Contributor Author

pires commented Aug 29, 2015

@jimmidyson do your best, I'm always eager to learn!

@jimmidyson
Copy link
Member

First impressions: really good work.

I'd like to separate out instructions into a simple cluster, i.e. just master & data nodes, for simple deployments that are easy to try out without knowing about different elasticsearch node roles. Then the more production ready clustering instructions that are already there. What do you think?

@pires
Copy link
Contributor Author

pires commented Aug 29, 2015

@jimmidyson I've gave some thought about that for a while - since I started my repo - and decided to keep it protip style. Why? Kubernetes cluster is a complex beast. Elasticsearch clusters are too. I for one can't commit to support people who don't understand the basics of either one.
I'm sorry. I know this may sound harsh or l33t, but personally have had enough of those experiences in the past, i.e. with the CoreOS documentation and script voodoo.

That said, I'm open to review any contributions that make this simpler.

@jimmidyson
Copy link
Member

While I understand your reasons, I think both a basic setup & a more advanced setup would be the best approach for the Elasticsearch example. The examples are there for a couple of things IMO: to guide new Kubernetes users through different styles of deployments as well as being able to try out Kubernetes on small machines/VMs. This example is great for those wanting a large Elasticsearch cluster, but let's also consider those just starting out, perhaps with a single node install for development cluster or small projects. This is just too heavy for them IMO. While your master/data/client node separation is best practice for larger clusters, there are also cases where a e.g. three node cluster consisting of multifunction nodes is perfectly suited to provide required resilience & performance.

While a Kubernetes cluster may be underneath a complex beast, every effort is made to make it as simple to operate as possible, as well as being suitable for clusters, both large & small. I feel that examples should aim to do the same.

@jimmidyson
Copy link
Member

Also, get rid of the deprecated dir - this is definitely the right direction to go in so I think we're safe getting rid of it.

@satnam6502
Copy link
Contributor

I think I discussed this with @pires in a Hangout -- I think it would be good to have two Elasticsearch examples in two different directories. One example would be tutorial in nature and its primary use would be to help explain concepts. A second example in the other directory could describe a more realistic setup. That way we could tease apart two conflicting requirements?

@jimmidyson
Copy link
Member

@satnam6502 You expressed it more clearly than I 👍 Exactly what I was thinking.

@pires
Copy link
Contributor Author

pires commented Aug 30, 2015

  • Removed the deprecated folder.
  • Split example in two parts, the basics and the production cluster

@satnam6502 @jimmidyson please review.

@pires
Copy link
Contributor Author

pires commented Aug 30, 2015

While maintaining CoreOS documentation, someone asked me to run the following command, but I don't understand what it does or why it's failing. Help?

$ hack/update-generated-docs.sh
+++ [0830 11:31:11] Building go targets for darwin/amd64:
    cmd/gendocs
    cmd/genman
    cmd/genbashcomp
    cmd/mungedocs
+++ [0830 11:31:12] Placing binaries
/Users/pires/Work/Projects/kubernetes/examples/elasticsearch/README.md
----
kubectl-dash-f:
51: target file "service-account.yaml" does not exist
52: target file "es-svc.yaml" does not exist
53: target file "es-rc.yaml" does not exist

/Users/pires/Work/Projects/kubernetes/examples/elasticsearch/production_cluster/README.md
----
kubectl-dash-f:
54: target file "service-account.yaml" does not exist
55: target file "es-discovery-svc.yaml" does not exist
56: target file "es-svc.yaml" does not exist
57: target file "es-master-rc.yaml" does not exist
63: target file "es-client-rc.yaml" does not exist
69: target file "es-data-rc.yaml" does not exist

FAIL: some manual changes are still required.
/Users/pires/Work/Projects/kubernetes/examples/ requires manual changes.  See preceding errors.
!!! Error in hack/update-generated-docs.sh:28
  '"${KUBE_ROOT}/hack/after-build/update-generated-docs.sh" "$@"' exited with status 1
Call stack:
  1: hack/update-generate

@brendandburns brendandburns assigned bparees and brendandburns and unassigned bparees Sep 1, 2015
@brendandburns
Copy link
Contributor

Assigning to myself as a reviewer since github acls don't allow @jimmidyson to be the reviewer for some reason.

Let me know when this is ok to merge...

@brendandburns brendandburns added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 1, 2015
@pires pires force-pushed the example_elasticsearch branch from 86b8fd9 to 1e6d702 Compare September 1, 2015 21:17
@jimmidyson
Copy link
Member

So reading the build logs for you 😜 :

hack/verify-flags-underscore.py
Found illegal 'flag' usage. If these are false positives you should run `hack/verify-flags-underscore.py -e > hack/verify-flags/exceptions.txt` to update the list.
examples/elasticsearch/README.md:  "cluster_name" : "myesdb",
examples/elasticsearch/README.md:  "cluster_name" : "myesdb",
examples/elasticsearch/production_cluster/README.md:  "cluster_name" : "myesdb",
examples/elasticsearch/production_cluster/README.md:  "cluster_name" : "myesdb",

Looks to me like the hack/verify-flags/exceptions.txt needs updating with the migrated cluster name.

@pires
Copy link
Contributor Author

pires commented Sep 3, 2015

This is ridiculous. @jimmidyson please come to my rescue (added you as collaborator) :/

P.S.: sorry but was on the phone. Where's the list of scripts we should run now? The docs munger thing worked pretty well, no errors.

@jimmidyson
Copy link
Member

I'm learning the same as you... will push soon to this PR branch for you to squash once it passes I hope!

@googlebot
Copy link

We found a Contributor License Agreement for you (the sender of this pull request) and all commit authors, but as best as we can tell these commits were authored by someone else. If that's the case, please add them to this pull request and have them confirm that they're okay with these commits being contributed to Google. If we're mistaken and you did author these commits, just reply here to confirm.

1 similar comment
@googlebot
Copy link

We found a Contributor License Agreement for you (the sender of this pull request) and all commit authors, but as best as we can tell these commits were authored by someone else. If that's the case, please add them to this pull request and have them confirm that they're okay with these commits being contributed to Google. If we're mistaken and you did author these commits, just reply here to confirm.

@googlebot googlebot added cla: no and removed cla: yes labels Sep 3, 2015
@jimmidyson
Copy link
Member

@pires Don't worry about the CLA thingy - once squashed you'll be sole author & it will go away.

@jimmidyson
Copy link
Member

@pires Shock we have green from travis!

@jimmidyson
Copy link
Member

@pires On to the next problem... see test failure in https://app.shippable.com/builds/55e8b2e8ef21fd0c000f1d36 Tests tab

@googlebot
Copy link

We found a Contributor License Agreement for you (the sender of this pull request) and all commit authors, but as best as we can tell these commits were authored by someone else. If that's the case, please add them to this pull request and have them confirm that they're okay with these commits being contributed to Google. If we're mistaken and you did author these commits, just reply here to confirm.

1 similar comment
@googlebot
Copy link

We found a Contributor License Agreement for you (the sender of this pull request) and all commit authors, but as best as we can tell these commits were authored by someone else. If that's the case, please add them to this pull request and have them confirm that they're okay with these commits being contributed to Google. If we're mistaken and you did author these commits, just reply here to confirm.

@jimmidyson
Copy link
Member

Updated examples tests... this is painful.

@jimmidyson
Copy link
Member

No way! Shippable passed too!!!!

@pires Please squash & push.

…r with JRE 8u51 and Elasticsearch 1.7.1.

Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
@pires pires force-pushed the example_elasticsearch branch from 3f303dc to 0a64995 Compare September 4, 2015 08:39
@googlebot googlebot added cla: yes and removed cla: no labels Sep 4, 2015
@pires
Copy link
Contributor Author

pires commented Sep 4, 2015

@jimmidyson you're my yang. I admire your patience. Anyway, squashed and pushed. Thank you so much!

@jimmidyson
Copy link
Member

@pires Happy to help.

@jimmidyson
Copy link
Member

Yay! @brendandburns LGTM

@a-robinson a-robinson added lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-merge labels Sep 4, 2015
@k8s-github-robot
Copy link

@k8s-bot ok to test

pr builder appears to be missing, activating due to 'lgtm' label.

@k8s-bot
Copy link

k8s-bot commented Sep 4, 2015

GCE e2e build/test passed for commit 0a64995.

ghost pushed a commit that referenced this pull request Sep 4, 2015
@ghost ghost merged commit e4fbfa9 into kubernetes:master Sep 4, 2015
@pires pires deleted the example_elasticsearch branch September 4, 2015 19:31
@jimmidyson
Copy link
Member

Thanks @pires for all your work on this.

@pires
Copy link
Contributor Author

pires commented Sep 4, 2015

@jimmidyson thank you for your help and guidance.

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.